{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Building a new chart in Splink\n",
    "\n",
    "As mentioned in the [Understanding Splink Charts topic guide](./understanding_and_editing_charts.md), splink charts are made up of three distinct parts:\n",
    "\n",
    "1. A function to create the dataset for the chart \n",
    "2. A template chart definition (in a json file)\n",
    "3. A function to read the chart definition, add the data to it, and return the chart itself \n",
    "\n",
    "## Worked Example\n",
    "\n",
    "Below is a worked example of how to create a new chart that shows all comparisons levels ordered by match weight:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "tags": [
     "hide_output"
    ]
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "You are using the default value for `max_pairs`, which may be too small and thus lead to inaccurate estimates for your model's u-parameters. Consider increasing to 1e8 or 1e9, which will result in more accurate estimates, but with a longer run time.\n",
      "----- Estimating u probabilities using random sampling -----\n",
      "\n",
      "Estimated u probabilities using random sampling\n",
      "\n",
      "Your model is not yet fully trained. Missing estimates for:\n",
      "    - first_name (no m values are trained).\n",
      "    - surname (no m values are trained).\n",
      "    - dob (no m values are trained).\n",
      "    - city (no m values are trained).\n",
      "    - email (no m values are trained).\n",
      "\n",
      "----- Starting EM training session -----\n",
      "\n",
      "Estimating the m probabilities of the model by blocking on:\n",
      "l.\"first_name\" = r.\"first_name\"\n",
      "\n",
      "Parameter estimates will be made for the following comparison(s):\n",
      "    - surname\n",
      "    - dob\n",
      "    - city\n",
      "    - email\n",
      "\n",
      "Parameter estimates cannot be made for the following comparison(s) since they are used in the blocking rules: \n",
      "    - first_name\n",
      "\n",
      "Iteration 1: Largest change in params was -0.39 in the m_probability of dob, level `Exact match on date of birth`\n",
      "Iteration 2: Largest change in params was -0.137 in the m_probability of dob, level `Exact match on date of birth`\n",
      "Iteration 3: Largest change in params was 0.0193 in the m_probability of dob, level `Abs date difference <= 10 year`\n",
      "Iteration 4: Largest change in params was 0.00568 in the m_probability of email, level `All other comparisons`\n",
      "Iteration 5: Largest change in params was 0.0016 in the m_probability of email, level `All other comparisons`\n",
      "Iteration 6: Largest change in params was 0.000447 in the m_probability of email, level `All other comparisons`\n",
      "Iteration 7: Largest change in params was 0.000127 in the m_probability of email, level `All other comparisons`\n",
      "Iteration 8: Largest change in params was 3.72e-05 in the m_probability of email, level `All other comparisons`\n",
      "\n",
      "EM converged after 8 iterations\n",
      "\n",
      "Your model is not yet fully trained. Missing estimates for:\n",
      "    - first_name (no m values are trained).\n",
      "\n",
      "----- Starting EM training session -----\n",
      "\n",
      "Estimating the m probabilities of the model by blocking on:\n",
      "l.\"dob\" = r.\"dob\"\n",
      "\n",
      "Parameter estimates will be made for the following comparison(s):\n",
      "    - first_name\n",
      "    - surname\n",
      "    - city\n",
      "    - email\n",
      "\n",
      "Parameter estimates cannot be made for the following comparison(s) since they are used in the blocking rules: \n",
      "    - dob\n",
      "\n",
      "Iteration 1: Largest change in params was 0.62 in probability_two_random_records_match\n",
      "Iteration 2: Largest change in params was 0.162 in probability_two_random_records_match\n",
      "Iteration 3: Largest change in params was 0.0821 in the m_probability of first_name, level `All other comparisons`\n",
      "Iteration 4: Largest change in params was 0.0311 in probability_two_random_records_match\n",
      "Iteration 5: Largest change in params was 0.0123 in probability_two_random_records_match\n",
      "Iteration 6: Largest change in params was 0.00543 in probability_two_random_records_match\n",
      "Iteration 7: Largest change in params was 0.00258 in probability_two_random_records_match\n",
      "Iteration 8: Largest change in params was 0.00128 in probability_two_random_records_match\n",
      "Iteration 9: Largest change in params was 0.000643 in probability_two_random_records_match\n",
      "Iteration 10: Largest change in params was 0.000328 in probability_two_random_records_match\n",
      "Iteration 11: Largest change in params was 0.000168 in probability_two_random_records_match\n",
      "Iteration 12: Largest change in params was 8.61e-05 in probability_two_random_records_match\n",
      "\n",
      "EM converged after 12 iterations\n",
      "\n",
      "Your model is fully trained. All comparisons have at least one estimate for their m and u values\n"
     ]
    }
   ],
   "source": [
    "import splink.comparison_library as cl\n",
    "from splink import DuckDBAPI, Linker, SettingsCreator, block_on, splink_datasets\n",
    "df = splink_datasets.fake_1000\n",
    "\n",
    "settings = SettingsCreator(\n",
    "    link_type=\"dedupe_only\",\n",
    "    comparisons=[\n",
    "      cl.NameComparison(\"first_name\"),\n",
    "        cl.NameComparison(\"surname\"),\n",
    "        cl.DateOfBirthComparison(\"dob\", input_is_string=True),\n",
    "        cl.ExactMatch(\"city\").configure(term_frequency_adjustments=True),\n",
    "        cl.LevenshteinAtThresholds(\"email\", 2),\n",
    "    ],\n",
    "    blocking_rules_to_generate_predictions=[\n",
    "        block_on(\"first_name\", \"dob\"),\n",
    "        block_on(\"surname\"),\n",
    "    ]\n",
    ")\n",
    "\n",
    "linker = Linker(df, settings,DuckDBAPI())\n",
    "linker.training.estimate_u_using_random_sampling(max_pairs=1e6)\n",
    "for rule in [block_on(\"first_name\"), block_on(\"dob\")]:\n",
    "    linker.training.estimate_parameters_using_expectation_maximisation(rule)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generate data for chart"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'comparison_name': 'first_name',\n",
       "  'sql_condition': '\"first_name_l\" = \"first_name_r\"',\n",
       "  'label_for_charts': 'Exact match on first_name',\n",
       "  'm_probability': 0.5009783629340309,\n",
       "  'u_probability': 0.0057935713975033705,\n",
       "  'bayes_factor': 86.4714229896119,\n",
       "  'log2_bayes_factor': 6.434151525637829,\n",
       "  'comparison_vector_value': 4},\n",
       " {'comparison_name': 'first_name',\n",
       "  'sql_condition': 'jaro_winkler_similarity(\"first_name_l\", \"first_name_r\") >= 0.92',\n",
       "  'label_for_charts': 'Jaro-Winkler distance of first_name >= 0.92',\n",
       "  'm_probability': 0.15450921411813767,\n",
       "  'u_probability': 0.0023429457903817435,\n",
       "  'bayes_factor': 65.9465595629351,\n",
       "  'log2_bayes_factor': 6.043225490816602,\n",
       "  'comparison_vector_value': 3},\n",
       " {'comparison_name': 'first_name',\n",
       "  'sql_condition': 'jaro_winkler_similarity(\"first_name_l\", \"first_name_r\") >= 0.88',\n",
       "  'label_for_charts': 'Jaro-Winkler distance of first_name >= 0.88',\n",
       "  'm_probability': 0.07548037415770431,\n",
       "  'u_probability': 0.0015484319951285285,\n",
       "  'bayes_factor': 48.7463281533646,\n",
       "  'log2_bayes_factor': 5.607221645966225,\n",
       "  'comparison_vector_value': 2}]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Take linker object and extract complete settings dict\n",
    "records = linker._settings_obj._parameters_as_detailed_records\n",
    "\n",
    "cols_to_keep = [\n",
    "    \"comparison_name\",\n",
    "    \"sql_condition\",\n",
    "    \"label_for_charts\",\n",
    "    \"m_probability\",\n",
    "    \"u_probability\",\n",
    "    \"bayes_factor\",\n",
    "    \"log2_bayes_factor\",\n",
    "    \"comparison_vector_value\"\n",
    "]\n",
    "\n",
    "# Keep useful information for a match weights chart\n",
    "records = [{k: r[k] for k in cols_to_keep}\n",
    "           for r in records\n",
    "           if r[\"comparison_vector_value\"] != -1 and r[\"comparison_sort_order\"] != -1]\n",
    "\n",
    "records[:3]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a chart template\n",
    "\n",
    "### Build prototype chart in Altair"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "<style>\n",
       "  #altair-viz-4206cc58153c44368929ea0277c13198.vega-embed {\n",
       "    width: 100%;\n",
       "    display: flex;\n",
       "  }\n",
       "\n",
       "  #altair-viz-4206cc58153c44368929ea0277c13198.vega-embed details,\n",
       "  #altair-viz-4206cc58153c44368929ea0277c13198.vega-embed details summary {\n",
       "    position: relative;\n",
       "  }\n",
       "</style>\n",
       "<div id=\"altair-viz-4206cc58153c44368929ea0277c13198\"></div>\n",
       "<script type=\"text/javascript\">\n",
       "  var VEGA_DEBUG = (typeof VEGA_DEBUG == \"undefined\") ? {} : VEGA_DEBUG;\n",
       "  (function(spec, embedOpt){\n",
       "    let outputDiv = document.currentScript.previousElementSibling;\n",
       "    if (outputDiv.id !== \"altair-viz-4206cc58153c44368929ea0277c13198\") {\n",
       "      outputDiv = document.getElementById(\"altair-viz-4206cc58153c44368929ea0277c13198\");\n",
       "    }\n",
       "    const paths = {\n",
       "      \"vega\": \"https://cdn.jsdelivr.net/npm/vega@5?noext\",\n",
       "      \"vega-lib\": \"https://cdn.jsdelivr.net/npm/vega-lib?noext\",\n",
       "      \"vega-lite\": \"https://cdn.jsdelivr.net/npm/vega-lite@5.17.0?noext\",\n",
       "      \"vega-embed\": \"https://cdn.jsdelivr.net/npm/vega-embed@6?noext\",\n",
       "    };\n",
       "\n",
       "    function maybeLoadScript(lib, version) {\n",
       "      var key = `${lib.replace(\"-\", \"\")}_version`;\n",
       "      return (VEGA_DEBUG[key] == version) ?\n",
       "        Promise.resolve(paths[lib]) :\n",
       "        new Promise(function(resolve, reject) {\n",
       "          var s = document.createElement('script');\n",
       "          document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
       "          s.async = true;\n",
       "          s.onload = () => {\n",
       "            VEGA_DEBUG[key] = version;\n",
       "            return resolve(paths[lib]);\n",
       "          };\n",
       "          s.onerror = () => reject(`Error loading script: ${paths[lib]}`);\n",
       "          s.src = paths[lib];\n",
       "        });\n",
       "    }\n",
       "\n",
       "    function showError(err) {\n",
       "      outputDiv.innerHTML = `<div class=\"error\" style=\"color:red;\">${err}</div>`;\n",
       "      throw err;\n",
       "    }\n",
       "\n",
       "    function displayChart(vegaEmbed) {\n",
       "      vegaEmbed(outputDiv, spec, embedOpt)\n",
       "        .catch(err => showError(`Javascript Error: ${err.message}<br>This usually means there's a typo in your chart specification. See the javascript console for the full traceback.`));\n",
       "    }\n",
       "\n",
       "    if(typeof define === \"function\" && define.amd) {\n",
       "      requirejs.config({paths});\n",
       "      require([\"vega-embed\"], displayChart, err => showError(`Error loading script: ${err.message}`));\n",
       "    } else {\n",
       "      maybeLoadScript(\"vega\", \"5\")\n",
       "        .then(() => maybeLoadScript(\"vega-lite\", \"5.17.0\"))\n",
       "        .then(() => maybeLoadScript(\"vega-embed\", \"6\"))\n",
       "        .catch(showError)\n",
       "        .then(() => displayChart(vegaEmbed));\n",
       "    }\n",
       "  })({\"config\": {\"view\": {\"continuousWidth\": 300, \"continuousHeight\": 300}}, \"data\": {\"name\": \"data-a32e879024788e12ff15754fbeed80f5\"}, \"mark\": {\"type\": \"bar\"}, \"encoding\": {\"color\": {\"field\": \"comparison_name\", \"type\": \"nominal\"}, \"x\": {\"field\": \"log2_bayes_factor\", \"type\": \"quantitative\"}, \"y\": {\"field\": \"cl_id\", \"type\": \"nominal\"}}, \"$schema\": \"https://vega.github.io/schema/vega-lite/v5.17.0.json\", \"datasets\": {\"data-a32e879024788e12ff15754fbeed80f5\": [{\"comparison_name\": \"first_name\", \"sql_condition\": \"\\\"first_name_l\\\" = \\\"first_name_r\\\"\", \"label_for_charts\": \"Exact match on first_name\", \"m_probability\": 0.5009783629340309, \"u_probability\": 0.0057935713975033705, \"bayes_factor\": 86.4714229896119, \"log2_bayes_factor\": 6.434151525637829, \"comparison_vector_value\": 4, \"cl_id\": \"first_name_4\"}, {\"comparison_name\": \"first_name\", \"sql_condition\": \"jaro_winkler_similarity(\\\"first_name_l\\\", \\\"first_name_r\\\") >= 0.92\", \"label_for_charts\": \"Jaro-Winkler distance of first_name >= 0.92\", \"m_probability\": 0.15450921411813767, \"u_probability\": 0.0023429457903817435, \"bayes_factor\": 65.9465595629351, \"log2_bayes_factor\": 6.043225490816602, \"comparison_vector_value\": 3, \"cl_id\": \"first_name_3\"}, {\"comparison_name\": \"first_name\", \"sql_condition\": \"jaro_winkler_similarity(\\\"first_name_l\\\", \\\"first_name_r\\\") >= 0.88\", \"label_for_charts\": \"Jaro-Winkler distance of first_name >= 0.88\", \"m_probability\": 0.07548037415770431, \"u_probability\": 0.0015484319951285285, \"bayes_factor\": 48.7463281533646, \"log2_bayes_factor\": 5.607221645966225, \"comparison_vector_value\": 2, \"cl_id\": \"first_name_2\"}, {\"comparison_name\": \"first_name\", \"sql_condition\": \"jaro_winkler_similarity(\\\"first_name_l\\\", \\\"first_name_r\\\") >= 0.7\", \"label_for_charts\": \"Jaro-Winkler distance of first_name >= 0.7\", \"m_probability\": 0.08040594405245684, \"u_probability\": 0.018934945558406913, \"bayes_factor\": 4.246431224448779, \"log2_bayes_factor\": 2.086250884145864, \"comparison_vector_value\": 1, \"cl_id\": \"first_name_1\"}, {\"comparison_name\": \"first_name\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.1886261047376702, \"u_probability\": 0.9713801052585794, \"bayes_factor\": 0.1941836194879226, \"log2_bayes_factor\": -2.36450658867333, \"comparison_vector_value\": 0, \"cl_id\": \"first_name_0\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"\\\"surname_l\\\" = \\\"surname_r\\\"\", \"label_for_charts\": \"Exact match on surname\", \"m_probability\": 0.5348631181016474, \"u_probability\": 0.004889975550122249, \"bayes_factor\": 109.3795076517869, \"log2_bayes_factor\": 6.77319866302027, \"comparison_vector_value\": 4, \"cl_id\": \"surname_4\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"jaro_winkler_similarity(\\\"surname_l\\\", \\\"surname_r\\\") >= 0.92\", \"label_for_charts\": \"Jaro-Winkler distance of surname >= 0.92\", \"m_probability\": 0.22996601178797366, \"u_probability\": 0.00283905173880724, \"bayes_factor\": 81.00099362211284, \"log2_bayes_factor\": 6.3398677002291, \"comparison_vector_value\": 3, \"cl_id\": \"surname_3\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"jaro_winkler_similarity(\\\"surname_l\\\", \\\"surname_r\\\") >= 0.88\", \"label_for_charts\": \"Jaro-Winkler distance of surname >= 0.88\", \"m_probability\": 0.04364798822195632, \"u_probability\": 0.0011314412292407403, \"bayes_factor\": 38.577335785480024, \"log2_bayes_factor\": 5.269681606812555, \"comparison_vector_value\": 2, \"cl_id\": \"surname_2\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"jaro_winkler_similarity(\\\"surname_l\\\", \\\"surname_r\\\") >= 0.7\", \"label_for_charts\": \"Jaro-Winkler distance of surname >= 0.7\", \"m_probability\": 0.08700065800936296, \"u_probability\": 0.014729633311540402, \"bayes_factor\": 5.906505353476757, \"log2_bayes_factor\": 2.562304797000941, \"comparison_vector_value\": 1, \"cl_id\": \"surname_1\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.10452222387905959, \"u_probability\": 0.9764098981702893, \"bayes_factor\": 0.10704748494963592, \"log2_bayes_factor\": -3.223677194477459, \"comparison_vector_value\": 0, \"cl_id\": \"surname_0\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"\\\"dob_l\\\" = \\\"dob_r\\\"\", \"label_for_charts\": \"Exact match on date of birth\", \"m_probability\": 0.4019265402159268, \"u_probability\": 0.0017477477477477479, \"bayes_factor\": 229.96827816478284, \"log2_bayes_factor\": 7.845291059246031, \"comparison_vector_value\": 5, \"cl_id\": \"dob_5\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"damerau_levenshtein(\\\"dob_l\\\", \\\"dob_r\\\") <= 1\", \"label_for_charts\": \"DamerauLevenshtein distance <= 1\", \"m_probability\": 0.1315183028543781, \"u_probability\": 0.0016436436436436436, \"bayes_factor\": 80.01631215074526, \"log2_bayes_factor\": 6.3222222331380395, \"comparison_vector_value\": 4, \"cl_id\": \"dob_4\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"ABS(EPOCH(try_strptime(\\\"dob_l\\\", '%Y-%m-%d')) - EPOCH(try_strptime(\\\"dob_r\\\", '%Y-%m-%d'))) <= 2629800.0\", \"label_for_charts\": \"Abs date difference <= 1 month\", \"m_probability\": 0.06377918999418847, \"u_probability\": 0.0024084084084084086, \"bayes_factor\": 26.48188312726279, \"log2_bayes_factor\": 4.726933810754774, \"comparison_vector_value\": 3, \"cl_id\": \"dob_3\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"ABS(EPOCH(try_strptime(\\\"dob_l\\\", '%Y-%m-%d')) - EPOCH(try_strptime(\\\"dob_r\\\", '%Y-%m-%d'))) <= 31557600.0\", \"label_for_charts\": \"Abs date difference <= 1 year\", \"m_probability\": 0.13931596112673353, \"u_probability\": 0.033053053053053054, \"bayes_factor\": 4.21491959920069, \"log2_bayes_factor\": 2.0755051116084613, \"comparison_vector_value\": 2, \"cl_id\": \"dob_2\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"ABS(EPOCH(try_strptime(\\\"dob_l\\\", '%Y-%m-%d')) - EPOCH(try_strptime(\\\"dob_r\\\", '%Y-%m-%d'))) <= 315576000.0\", \"label_for_charts\": \"Abs date difference <= 10 year\", \"m_probability\": 0.21851691590096683, \"u_probability\": 0.30747947947947946, \"bayes_factor\": 0.7106715422794587, \"log2_bayes_factor\": -0.49274516487791215, \"comparison_vector_value\": 1, \"cl_id\": \"dob_1\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.04494308990780629, \"u_probability\": 0.6536676676676677, \"bayes_factor\": 0.06875525917958647, \"log2_bayes_factor\": -3.8623861184011, \"comparison_vector_value\": 0, \"cl_id\": \"dob_0\"}, {\"comparison_name\": \"city\", \"sql_condition\": \"\\\"city_l\\\" = \\\"city_r\\\"\", \"label_for_charts\": \"Exact match on city\", \"m_probability\": 0.5648986160568799, \"u_probability\": 0.0551475711801453, \"bayes_factor\": 10.243399735803042, \"log2_bayes_factor\": 3.3566227133460154, \"comparison_vector_value\": 1, \"cl_id\": \"city_1\"}, {\"comparison_name\": \"city\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.43510138394312015, \"u_probability\": 0.9448524288198547, \"bayes_factor\": 0.4604966560614901, \"log2_bayes_factor\": -1.1187374147944675, \"comparison_vector_value\": 0, \"cl_id\": \"city_0\"}, {\"comparison_name\": \"email\", \"sql_condition\": \"\\\"email_l\\\" = \\\"email_r\\\"\", \"label_for_charts\": \"Exact match on email\", \"m_probability\": 0.5728567965549329, \"u_probability\": 0.0021938713143283602, \"bayes_factor\": 261.1168635159029, \"log2_bayes_factor\": 8.028551822933485, \"comparison_vector_value\": 2, \"cl_id\": \"email_2\"}, {\"comparison_name\": \"email\", \"sql_condition\": \"levenshtein(\\\"email_l\\\", \\\"email_r\\\") <= 2\", \"label_for_charts\": \"Levenshtein distance of email <= 2\", \"m_probability\": 0.3102931452757915, \"u_probability\": 0.0013542812658830492, \"bayes_factor\": 229.12016365630453, \"log2_bayes_factor\": 7.839960617980343, \"comparison_vector_value\": 1, \"cl_id\": \"email_1\"}, {\"comparison_name\": \"email\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.11685005816927549, \"u_probability\": 0.9964518474197885, \"bayes_factor\": 0.11726613631340733, \"log2_bayes_factor\": -3.092141637583301, \"comparison_vector_value\": 0, \"cl_id\": \"email_0\"}]}}, {\"mode\": \"vega-lite\"});\n",
       "</script>"
      ],
      "text/plain": [
       "alt.Chart(...)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import altair as alt\n",
    "\n",
    "df = pd.DataFrame(records)\n",
    "\n",
    "# Need a unique name for each comparison level - easier to create in pandas than altair\n",
    "df[\"cl_id\"] = df[\"comparison_name\"] + \"_\" + \\\n",
    "    df[\"comparison_vector_value\"].astype(\"str\")\n",
    "\n",
    "# Simple start - bar chart with x, y and color encodings\n",
    "alt.Chart(df).mark_bar().encode(\n",
    "    y=\"cl_id\",\n",
    "    x=\"log2_bayes_factor\",\n",
    "    color=\"comparison_name\"\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Sort bars, edit axes/titles\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "<style>\n",
       "  #altair-viz-facb831b7d2a4664be840a654a392fc1.vega-embed {\n",
       "    width: 100%;\n",
       "    display: flex;\n",
       "  }\n",
       "\n",
       "  #altair-viz-facb831b7d2a4664be840a654a392fc1.vega-embed details,\n",
       "  #altair-viz-facb831b7d2a4664be840a654a392fc1.vega-embed details summary {\n",
       "    position: relative;\n",
       "  }\n",
       "</style>\n",
       "<div id=\"altair-viz-facb831b7d2a4664be840a654a392fc1\"></div>\n",
       "<script type=\"text/javascript\">\n",
       "  var VEGA_DEBUG = (typeof VEGA_DEBUG == \"undefined\") ? {} : VEGA_DEBUG;\n",
       "  (function(spec, embedOpt){\n",
       "    let outputDiv = document.currentScript.previousElementSibling;\n",
       "    if (outputDiv.id !== \"altair-viz-facb831b7d2a4664be840a654a392fc1\") {\n",
       "      outputDiv = document.getElementById(\"altair-viz-facb831b7d2a4664be840a654a392fc1\");\n",
       "    }\n",
       "    const paths = {\n",
       "      \"vega\": \"https://cdn.jsdelivr.net/npm/vega@5?noext\",\n",
       "      \"vega-lib\": \"https://cdn.jsdelivr.net/npm/vega-lib?noext\",\n",
       "      \"vega-lite\": \"https://cdn.jsdelivr.net/npm/vega-lite@5.8.0?noext\",\n",
       "      \"vega-embed\": \"https://cdn.jsdelivr.net/npm/vega-embed@6?noext\",\n",
       "    };\n",
       "\n",
       "    function maybeLoadScript(lib, version) {\n",
       "      var key = `${lib.replace(\"-\", \"\")}_version`;\n",
       "      return (VEGA_DEBUG[key] == version) ?\n",
       "        Promise.resolve(paths[lib]) :\n",
       "        new Promise(function(resolve, reject) {\n",
       "          var s = document.createElement('script');\n",
       "          document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
       "          s.async = true;\n",
       "          s.onload = () => {\n",
       "            VEGA_DEBUG[key] = version;\n",
       "            return resolve(paths[lib]);\n",
       "          };\n",
       "          s.onerror = () => reject(`Error loading script: ${paths[lib]}`);\n",
       "          s.src = paths[lib];\n",
       "        });\n",
       "    }\n",
       "\n",
       "    function showError(err) {\n",
       "      outputDiv.innerHTML = `<div class=\"error\" style=\"color:red;\">${err}</div>`;\n",
       "      throw err;\n",
       "    }\n",
       "\n",
       "    function displayChart(vegaEmbed) {\n",
       "      vegaEmbed(outputDiv, spec, embedOpt)\n",
       "        .catch(err => showError(`Javascript Error: ${err.message}<br>This usually means there's a typo in your chart specification. See the javascript console for the full traceback.`));\n",
       "    }\n",
       "\n",
       "    if(typeof define === \"function\" && define.amd) {\n",
       "      requirejs.config({paths});\n",
       "      require([\"vega-embed\"], displayChart, err => showError(`Error loading script: ${err.message}`));\n",
       "    } else {\n",
       "      maybeLoadScript(\"vega\", \"5\")\n",
       "        .then(() => maybeLoadScript(\"vega-lite\", \"5.8.0\"))\n",
       "        .then(() => maybeLoadScript(\"vega-embed\", \"6\"))\n",
       "        .catch(showError)\n",
       "        .then(() => displayChart(vegaEmbed));\n",
       "    }\n",
       "  })({\"config\": {\"view\": {\"continuousWidth\": 300, \"continuousHeight\": 300, \"step\": 15}}, \"data\": {\"name\": \"data-3901c03d78701611834aa82ab7374cce\"}, \"mark\": {\"type\": \"bar\"}, \"encoding\": {\"color\": {\"field\": \"comparison_name\", \"type\": \"nominal\"}, \"x\": {\"field\": \"log2_bayes_factor\", \"scale\": {\"domain\": [-10, 10]}, \"title\": \"Comparison level match weight = log2(m/u)\", \"type\": \"quantitative\"}, \"y\": {\"field\": \"cl_id\", \"sort\": \"-x\", \"title\": \"Comparison level\", \"type\": \"nominal\"}}, \"title\": \"New Chart - WOO!\", \"$schema\": \"https://vega.github.io/schema/vega-lite/v5.8.0.json\", \"datasets\": {\"data-3901c03d78701611834aa82ab7374cce\": [{\"comparison_name\": \"first_name\", \"sql_condition\": \"\\\"first_name_l\\\" = \\\"first_name_r\\\"\", \"label_for_charts\": \"Exact match first_name\", \"m_probability\": 0.5018941916173814, \"u_probability\": 0.0057935713975033705, \"bayes_factor\": 86.62949969575988, \"log2_bayes_factor\": 6.436786480320881, \"comparison_vector_value\": 4, \"cl_id\": \"first_name_4\"}, {\"comparison_name\": \"first_name\", \"sql_condition\": \"damerau_levenshtein(\\\"first_name_l\\\", \\\"first_name_r\\\") <= 1\", \"label_for_charts\": \"Damerau_levenshtein <= 1\", \"m_probability\": 0.19595791797531015, \"u_probability\": 0.00236614327345483, \"bayes_factor\": 82.81743551783742, \"log2_bayes_factor\": 6.371862624533329, \"comparison_vector_value\": 3, \"cl_id\": \"first_name_3\"}, {\"comparison_name\": \"first_name\", \"sql_condition\": \"jaro_winkler_similarity(\\\"first_name_l\\\", \\\"first_name_r\\\") >= 0.9\", \"label_for_charts\": \"Jaro_winkler_similarity >= 0.9\", \"m_probability\": 0.045985303626033085, \"u_probability\": 0.001296159366708712, \"bayes_factor\": 35.47812468678278, \"log2_bayes_factor\": 5.148857848140163, \"comparison_vector_value\": 2, \"cl_id\": \"first_name_2\"}, {\"comparison_name\": \"first_name\", \"sql_condition\": \"jaro_winkler_similarity(\\\"first_name_l\\\", \\\"first_name_r\\\") >= 0.8\", \"label_for_charts\": \"Jaro_winkler_similarity >= 0.8\", \"m_probability\": 0.06396730257493154, \"u_probability\": 0.005677583982137938, \"bayes_factor\": 11.266641370022352, \"log2_bayes_factor\": 3.493985601438375, \"comparison_vector_value\": 1, \"cl_id\": \"first_name_1\"}, {\"comparison_name\": \"first_name\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.19219528420634394, \"u_probability\": 0.9848665419801952, \"bayes_factor\": 0.19514855669673956, \"log2_bayes_factor\": -2.357355302129234, \"comparison_vector_value\": 0, \"cl_id\": \"first_name_0\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"\\\"surname_l\\\" = \\\"surname_r\\\"\", \"label_for_charts\": \"Exact match surname\", \"m_probability\": 0.5527050424941531, \"u_probability\": 0.004889975550122249, \"bayes_factor\": 113.02818119005431, \"log2_bayes_factor\": 6.820538712806792, \"comparison_vector_value\": 4, \"cl_id\": \"surname_4\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"damerau_levenshtein(\\\"surname_l\\\", \\\"surname_r\\\") <= 1\", \"label_for_charts\": \"Damerau_levenshtein <= 1\", \"m_probability\": 0.22212752320956386, \"u_probability\": 0.0027554624131641246, \"bayes_factor\": 80.61351958508214, \"log2_bayes_factor\": 6.332949906378981, \"comparison_vector_value\": 3, \"cl_id\": \"surname_3\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"jaro_winkler_similarity(\\\"surname_l\\\", \\\"surname_r\\\") >= 0.9\", \"label_for_charts\": \"Jaro_winkler_similarity >= 0.9\", \"m_probability\": 0.0490149338194711, \"u_probability\": 0.0010090425738347498, \"bayes_factor\": 48.57568460485815, \"log2_bayes_factor\": 5.602162423566203, \"comparison_vector_value\": 2, \"cl_id\": \"surname_2\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"jaro_winkler_similarity(\\\"surname_l\\\", \\\"surname_r\\\") >= 0.8\", \"label_for_charts\": \"Jaro_winkler_similarity >= 0.8\", \"m_probability\": 0.05001678986356945, \"u_probability\": 0.003710768991942586, \"bayes_factor\": 13.478820689774516, \"log2_bayes_factor\": 3.752622370380284, \"comparison_vector_value\": 1, \"cl_id\": \"surname_1\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.1261357106132424, \"u_probability\": 0.9876347504709363, \"bayes_factor\": 0.1277149376863226, \"log2_bayes_factor\": -2.969000820703079, \"comparison_vector_value\": 0, \"cl_id\": \"surname_0\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"\\\"dob_l\\\" = \\\"dob_r\\\"\", \"label_for_charts\": \"Exact match\", \"m_probability\": 0.41383785481447766, \"u_probability\": 0.0017477477477477479, \"bayes_factor\": 236.78351486807742, \"log2_bayes_factor\": 7.887424832202931, \"comparison_vector_value\": 5, \"cl_id\": \"dob_5\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"damerau_levenshtein(\\\"dob_l\\\", \\\"dob_r\\\") <= 1\", \"label_for_charts\": \"Damerau_levenshtein <= 1\", \"m_probability\": 0.10806341031654734, \"u_probability\": 0.0016436436436436436, \"bayes_factor\": 65.74625268345359, \"log2_bayes_factor\": 6.038836762842662, \"comparison_vector_value\": 4, \"cl_id\": \"dob_4\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"\\n            abs(date_diff('month',\\n                strptime(\\\"dob_l\\\", '%Y-%m-%d'),\\n                strptime(\\\"dob_r\\\", '%Y-%m-%d'))\\n                ) <= 1\\n        \", \"label_for_charts\": \"Within 1 month\", \"m_probability\": 0.11300938544779224, \"u_probability\": 0.003833833833833834, \"bayes_factor\": 29.476860590690453, \"log2_bayes_factor\": 4.881510974428093, \"comparison_vector_value\": 3, \"cl_id\": \"dob_3\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"\\n            abs(date_diff('year',\\n                strptime(\\\"dob_l\\\", '%Y-%m-%d'),\\n                strptime(\\\"dob_r\\\", '%Y-%m-%d'))\\n                ) <= 1\\n        \", \"label_for_charts\": \"Within 1 year\", \"m_probability\": 0.17200656922328977, \"u_probability\": 0.05062662662662663, \"bayes_factor\": 3.397551460259144, \"log2_bayes_factor\": 1.7644954026183992, \"comparison_vector_value\": 2, \"cl_id\": \"dob_2\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"\\n            abs(date_diff('year',\\n                strptime(\\\"dob_l\\\", '%Y-%m-%d'),\\n                strptime(\\\"dob_r\\\", '%Y-%m-%d'))\\n                ) <= 10\\n        \", \"label_for_charts\": \"Within 10 years\", \"m_probability\": 0.19035523041792068, \"u_probability\": 0.3037037037037037, \"bayes_factor\": 0.6267794172297388, \"log2_bayes_factor\": -0.6739702908716182, \"comparison_vector_value\": 1, \"cl_id\": \"dob_1\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.002727549779972325, \"u_probability\": 0.6384444444444445, \"bayes_factor\": 0.004272180302776005, \"log2_bayes_factor\": -7.870811748958801, \"comparison_vector_value\": 0, \"cl_id\": \"dob_0\"}, {\"comparison_name\": \"city\", \"sql_condition\": \"\\\"city_l\\\" = \\\"city_r\\\"\", \"label_for_charts\": \"Exact match\", \"m_probability\": 0.6013808934279701, \"u_probability\": 0.0551475711801453, \"bayes_factor\": 10.904938885948333, \"log2_bayes_factor\": 3.4469097796586596, \"comparison_vector_value\": 1, \"cl_id\": \"city_1\"}, {\"comparison_name\": \"city\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.3986191065720299, \"u_probability\": 0.9448524288198547, \"bayes_factor\": 0.42188504195296994, \"log2_bayes_factor\": -1.2450781575619725, \"comparison_vector_value\": 0, \"cl_id\": \"city_0\"}, {\"comparison_name\": \"email\", \"sql_condition\": \"\\\"email_l\\\" = \\\"email_r\\\"\", \"label_for_charts\": \"Exact match\", \"m_probability\": 0.5914840252879943, \"u_probability\": 0.0021938713143283602, \"bayes_factor\": 269.6074384240141, \"log2_bayes_factor\": 8.07471649055784, \"comparison_vector_value\": 2, \"cl_id\": \"email_2\"}, {\"comparison_name\": \"email\", \"sql_condition\": \"levenshtein(\\\"email_l\\\", \\\"email_r\\\") <= 2\", \"label_for_charts\": \"Levenshtein <= 2\", \"m_probability\": 0.3019669634613132, \"u_probability\": 0.0013542812658830492, \"bayes_factor\": 222.9721189153553, \"log2_bayes_factor\": 7.800719512398763, \"comparison_vector_value\": 1, \"cl_id\": \"email_1\"}, {\"comparison_name\": \"email\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.10654901125069259, \"u_probability\": 0.9964518474197885, \"bayes_factor\": 0.10692840956298139, \"log2_bayes_factor\": -3.225282884575804, \"comparison_vector_value\": 0, \"cl_id\": \"email_0\"}]}}, {\"mode\": \"vega-lite\"});\n",
       "</script>"
      ],
      "text/plain": [
       "alt.Chart(...)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "alt.Chart(df).mark_bar().encode(\n",
    "    y=alt.Y(\"cl_id\",\n",
    "        sort=\"-x\",\n",
    "        title=\"Comparison level\"\n",
    "    ),\n",
    "    x=alt.X(\"log2_bayes_factor\",\n",
    "        title=\"Comparison level match weight = log2(m/u)\",\n",
    "        scale=alt.Scale(domain=[-10,10])\n",
    "    ),\n",
    "    color=\"comparison_name\"\n",
    ").properties(\n",
    "    title=\"New Chart - WOO!\"\n",
    ").configure_view(\n",
    "    step=15\n",
    ")\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Add tooltip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "<style>\n",
       "  #altair-viz-0a4d51676a1c4980aed997c689140668.vega-embed {\n",
       "    width: 100%;\n",
       "    display: flex;\n",
       "  }\n",
       "\n",
       "  #altair-viz-0a4d51676a1c4980aed997c689140668.vega-embed details,\n",
       "  #altair-viz-0a4d51676a1c4980aed997c689140668.vega-embed details summary {\n",
       "    position: relative;\n",
       "  }\n",
       "</style>\n",
       "<div id=\"altair-viz-0a4d51676a1c4980aed997c689140668\"></div>\n",
       "<script type=\"text/javascript\">\n",
       "  var VEGA_DEBUG = (typeof VEGA_DEBUG == \"undefined\") ? {} : VEGA_DEBUG;\n",
       "  (function(spec, embedOpt){\n",
       "    let outputDiv = document.currentScript.previousElementSibling;\n",
       "    if (outputDiv.id !== \"altair-viz-0a4d51676a1c4980aed997c689140668\") {\n",
       "      outputDiv = document.getElementById(\"altair-viz-0a4d51676a1c4980aed997c689140668\");\n",
       "    }\n",
       "    const paths = {\n",
       "      \"vega\": \"https://cdn.jsdelivr.net/npm/vega@5?noext\",\n",
       "      \"vega-lib\": \"https://cdn.jsdelivr.net/npm/vega-lib?noext\",\n",
       "      \"vega-lite\": \"https://cdn.jsdelivr.net/npm/vega-lite@5.8.0?noext\",\n",
       "      \"vega-embed\": \"https://cdn.jsdelivr.net/npm/vega-embed@6?noext\",\n",
       "    };\n",
       "\n",
       "    function maybeLoadScript(lib, version) {\n",
       "      var key = `${lib.replace(\"-\", \"\")}_version`;\n",
       "      return (VEGA_DEBUG[key] == version) ?\n",
       "        Promise.resolve(paths[lib]) :\n",
       "        new Promise(function(resolve, reject) {\n",
       "          var s = document.createElement('script');\n",
       "          document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
       "          s.async = true;\n",
       "          s.onload = () => {\n",
       "            VEGA_DEBUG[key] = version;\n",
       "            return resolve(paths[lib]);\n",
       "          };\n",
       "          s.onerror = () => reject(`Error loading script: ${paths[lib]}`);\n",
       "          s.src = paths[lib];\n",
       "        });\n",
       "    }\n",
       "\n",
       "    function showError(err) {\n",
       "      outputDiv.innerHTML = `<div class=\"error\" style=\"color:red;\">${err}</div>`;\n",
       "      throw err;\n",
       "    }\n",
       "\n",
       "    function displayChart(vegaEmbed) {\n",
       "      vegaEmbed(outputDiv, spec, embedOpt)\n",
       "        .catch(err => showError(`Javascript Error: ${err.message}<br>This usually means there's a typo in your chart specification. See the javascript console for the full traceback.`));\n",
       "    }\n",
       "\n",
       "    if(typeof define === \"function\" && define.amd) {\n",
       "      requirejs.config({paths});\n",
       "      require([\"vega-embed\"], displayChart, err => showError(`Error loading script: ${err.message}`));\n",
       "    } else {\n",
       "      maybeLoadScript(\"vega\", \"5\")\n",
       "        .then(() => maybeLoadScript(\"vega-lite\", \"5.8.0\"))\n",
       "        .then(() => maybeLoadScript(\"vega-embed\", \"6\"))\n",
       "        .catch(showError)\n",
       "        .then(() => displayChart(vegaEmbed));\n",
       "    }\n",
       "  })({\"config\": {\"view\": {\"continuousWidth\": 300, \"continuousHeight\": 300, \"step\": 15}}, \"data\": {\"name\": \"data-3901c03d78701611834aa82ab7374cce\"}, \"mark\": {\"type\": \"bar\"}, \"encoding\": {\"color\": {\"field\": \"comparison_name\", \"type\": \"nominal\"}, \"tooltip\": [{\"field\": \"comparison_name\", \"type\": \"nominal\"}, {\"field\": \"label_for_charts\", \"type\": \"nominal\"}, {\"field\": \"sql_condition\", \"type\": \"nominal\"}, {\"field\": \"m_probability\", \"type\": \"quantitative\"}, {\"field\": \"u_probability\", \"type\": \"quantitative\"}, {\"field\": \"bayes_factor\", \"type\": \"quantitative\"}, {\"field\": \"log2_bayes_factor\", \"type\": \"quantitative\"}], \"x\": {\"field\": \"log2_bayes_factor\", \"scale\": {\"domain\": [-10, 10]}, \"title\": \"Comparison level match weight = log2(m/u)\", \"type\": \"quantitative\"}, \"y\": {\"field\": \"cl_id\", \"sort\": \"-x\", \"title\": \"Comparison level\", \"type\": \"nominal\"}}, \"title\": \"New Chart - WOO!\", \"$schema\": \"https://vega.github.io/schema/vega-lite/v5.8.0.json\", \"datasets\": {\"data-3901c03d78701611834aa82ab7374cce\": [{\"comparison_name\": \"first_name\", \"sql_condition\": \"\\\"first_name_l\\\" = \\\"first_name_r\\\"\", \"label_for_charts\": \"Exact match first_name\", \"m_probability\": 0.5018941916173814, \"u_probability\": 0.0057935713975033705, \"bayes_factor\": 86.62949969575988, \"log2_bayes_factor\": 6.436786480320881, \"comparison_vector_value\": 4, \"cl_id\": \"first_name_4\"}, {\"comparison_name\": \"first_name\", \"sql_condition\": \"damerau_levenshtein(\\\"first_name_l\\\", \\\"first_name_r\\\") <= 1\", \"label_for_charts\": \"Damerau_levenshtein <= 1\", \"m_probability\": 0.19595791797531015, \"u_probability\": 0.00236614327345483, \"bayes_factor\": 82.81743551783742, \"log2_bayes_factor\": 6.371862624533329, \"comparison_vector_value\": 3, \"cl_id\": \"first_name_3\"}, {\"comparison_name\": \"first_name\", \"sql_condition\": \"jaro_winkler_similarity(\\\"first_name_l\\\", \\\"first_name_r\\\") >= 0.9\", \"label_for_charts\": \"Jaro_winkler_similarity >= 0.9\", \"m_probability\": 0.045985303626033085, \"u_probability\": 0.001296159366708712, \"bayes_factor\": 35.47812468678278, \"log2_bayes_factor\": 5.148857848140163, \"comparison_vector_value\": 2, \"cl_id\": \"first_name_2\"}, {\"comparison_name\": \"first_name\", \"sql_condition\": \"jaro_winkler_similarity(\\\"first_name_l\\\", \\\"first_name_r\\\") >= 0.8\", \"label_for_charts\": \"Jaro_winkler_similarity >= 0.8\", \"m_probability\": 0.06396730257493154, \"u_probability\": 0.005677583982137938, \"bayes_factor\": 11.266641370022352, \"log2_bayes_factor\": 3.493985601438375, \"comparison_vector_value\": 1, \"cl_id\": \"first_name_1\"}, {\"comparison_name\": \"first_name\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.19219528420634394, \"u_probability\": 0.9848665419801952, \"bayes_factor\": 0.19514855669673956, \"log2_bayes_factor\": -2.357355302129234, \"comparison_vector_value\": 0, \"cl_id\": \"first_name_0\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"\\\"surname_l\\\" = \\\"surname_r\\\"\", \"label_for_charts\": \"Exact match surname\", \"m_probability\": 0.5527050424941531, \"u_probability\": 0.004889975550122249, \"bayes_factor\": 113.02818119005431, \"log2_bayes_factor\": 6.820538712806792, \"comparison_vector_value\": 4, \"cl_id\": \"surname_4\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"damerau_levenshtein(\\\"surname_l\\\", \\\"surname_r\\\") <= 1\", \"label_for_charts\": \"Damerau_levenshtein <= 1\", \"m_probability\": 0.22212752320956386, \"u_probability\": 0.0027554624131641246, \"bayes_factor\": 80.61351958508214, \"log2_bayes_factor\": 6.332949906378981, \"comparison_vector_value\": 3, \"cl_id\": \"surname_3\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"jaro_winkler_similarity(\\\"surname_l\\\", \\\"surname_r\\\") >= 0.9\", \"label_for_charts\": \"Jaro_winkler_similarity >= 0.9\", \"m_probability\": 0.0490149338194711, \"u_probability\": 0.0010090425738347498, \"bayes_factor\": 48.57568460485815, \"log2_bayes_factor\": 5.602162423566203, \"comparison_vector_value\": 2, \"cl_id\": \"surname_2\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"jaro_winkler_similarity(\\\"surname_l\\\", \\\"surname_r\\\") >= 0.8\", \"label_for_charts\": \"Jaro_winkler_similarity >= 0.8\", \"m_probability\": 0.05001678986356945, \"u_probability\": 0.003710768991942586, \"bayes_factor\": 13.478820689774516, \"log2_bayes_factor\": 3.752622370380284, \"comparison_vector_value\": 1, \"cl_id\": \"surname_1\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.1261357106132424, \"u_probability\": 0.9876347504709363, \"bayes_factor\": 0.1277149376863226, \"log2_bayes_factor\": -2.969000820703079, \"comparison_vector_value\": 0, \"cl_id\": \"surname_0\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"\\\"dob_l\\\" = \\\"dob_r\\\"\", \"label_for_charts\": \"Exact match\", \"m_probability\": 0.41383785481447766, \"u_probability\": 0.0017477477477477479, \"bayes_factor\": 236.78351486807742, \"log2_bayes_factor\": 7.887424832202931, \"comparison_vector_value\": 5, \"cl_id\": \"dob_5\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"damerau_levenshtein(\\\"dob_l\\\", \\\"dob_r\\\") <= 1\", \"label_for_charts\": \"Damerau_levenshtein <= 1\", \"m_probability\": 0.10806341031654734, \"u_probability\": 0.0016436436436436436, \"bayes_factor\": 65.74625268345359, \"log2_bayes_factor\": 6.038836762842662, \"comparison_vector_value\": 4, \"cl_id\": \"dob_4\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"\\n            abs(date_diff('month',\\n                strptime(\\\"dob_l\\\", '%Y-%m-%d'),\\n                strptime(\\\"dob_r\\\", '%Y-%m-%d'))\\n                ) <= 1\\n        \", \"label_for_charts\": \"Within 1 month\", \"m_probability\": 0.11300938544779224, \"u_probability\": 0.003833833833833834, \"bayes_factor\": 29.476860590690453, \"log2_bayes_factor\": 4.881510974428093, \"comparison_vector_value\": 3, \"cl_id\": \"dob_3\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"\\n            abs(date_diff('year',\\n                strptime(\\\"dob_l\\\", '%Y-%m-%d'),\\n                strptime(\\\"dob_r\\\", '%Y-%m-%d'))\\n                ) <= 1\\n        \", \"label_for_charts\": \"Within 1 year\", \"m_probability\": 0.17200656922328977, \"u_probability\": 0.05062662662662663, \"bayes_factor\": 3.397551460259144, \"log2_bayes_factor\": 1.7644954026183992, \"comparison_vector_value\": 2, \"cl_id\": \"dob_2\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"\\n            abs(date_diff('year',\\n                strptime(\\\"dob_l\\\", '%Y-%m-%d'),\\n                strptime(\\\"dob_r\\\", '%Y-%m-%d'))\\n                ) <= 10\\n        \", \"label_for_charts\": \"Within 10 years\", \"m_probability\": 0.19035523041792068, \"u_probability\": 0.3037037037037037, \"bayes_factor\": 0.6267794172297388, \"log2_bayes_factor\": -0.6739702908716182, \"comparison_vector_value\": 1, \"cl_id\": \"dob_1\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.002727549779972325, \"u_probability\": 0.6384444444444445, \"bayes_factor\": 0.004272180302776005, \"log2_bayes_factor\": -7.870811748958801, \"comparison_vector_value\": 0, \"cl_id\": \"dob_0\"}, {\"comparison_name\": \"city\", \"sql_condition\": \"\\\"city_l\\\" = \\\"city_r\\\"\", \"label_for_charts\": \"Exact match\", \"m_probability\": 0.6013808934279701, \"u_probability\": 0.0551475711801453, \"bayes_factor\": 10.904938885948333, \"log2_bayes_factor\": 3.4469097796586596, \"comparison_vector_value\": 1, \"cl_id\": \"city_1\"}, {\"comparison_name\": \"city\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.3986191065720299, \"u_probability\": 0.9448524288198547, \"bayes_factor\": 0.42188504195296994, \"log2_bayes_factor\": -1.2450781575619725, \"comparison_vector_value\": 0, \"cl_id\": \"city_0\"}, {\"comparison_name\": \"email\", \"sql_condition\": \"\\\"email_l\\\" = \\\"email_r\\\"\", \"label_for_charts\": \"Exact match\", \"m_probability\": 0.5914840252879943, \"u_probability\": 0.0021938713143283602, \"bayes_factor\": 269.6074384240141, \"log2_bayes_factor\": 8.07471649055784, \"comparison_vector_value\": 2, \"cl_id\": \"email_2\"}, {\"comparison_name\": \"email\", \"sql_condition\": \"levenshtein(\\\"email_l\\\", \\\"email_r\\\") <= 2\", \"label_for_charts\": \"Levenshtein <= 2\", \"m_probability\": 0.3019669634613132, \"u_probability\": 0.0013542812658830492, \"bayes_factor\": 222.9721189153553, \"log2_bayes_factor\": 7.800719512398763, \"comparison_vector_value\": 1, \"cl_id\": \"email_1\"}, {\"comparison_name\": \"email\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.10654901125069259, \"u_probability\": 0.9964518474197885, \"bayes_factor\": 0.10692840956298139, \"log2_bayes_factor\": -3.225282884575804, \"comparison_vector_value\": 0, \"cl_id\": \"email_0\"}]}}, {\"mode\": \"vega-lite\"});\n",
       "</script>"
      ],
      "text/plain": [
       "alt.Chart(...)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "alt.Chart(df).mark_bar().encode(\n",
    "    y=alt.Y(\"cl_id\",\n",
    "            sort=\"-x\",\n",
    "            title=\"Comparison level\"\n",
    "            ),\n",
    "    x=alt.X(\"log2_bayes_factor\",\n",
    "            title=\"Comparison level match weight = log2(m/u)\",\n",
    "            scale=alt.Scale(domain=[-10, 10])\n",
    "            ),\n",
    "    color=\"comparison_name\",\n",
    "    tooltip=[\n",
    "        \"comparison_name\",\n",
    "        \"label_for_charts\",\n",
    "        \"sql_condition\",\n",
    "        \"m_probability\",\n",
    "        \"u_probability\",\n",
    "        \"bayes_factor\",\n",
    "        \"log2_bayes_factor\"\n",
    "        ]\n",
    ").properties(\n",
    "    title=\"New Chart - WOO!\"\n",
    ").configure_view(\n",
    "    step=15\n",
    ")\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Add text layer\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "<style>\n",
       "  #altair-viz-d09af60bd6344262a6b3c497f86ff49a.vega-embed {\n",
       "    width: 100%;\n",
       "    display: flex;\n",
       "  }\n",
       "\n",
       "  #altair-viz-d09af60bd6344262a6b3c497f86ff49a.vega-embed details,\n",
       "  #altair-viz-d09af60bd6344262a6b3c497f86ff49a.vega-embed details summary {\n",
       "    position: relative;\n",
       "  }\n",
       "</style>\n",
       "<div id=\"altair-viz-d09af60bd6344262a6b3c497f86ff49a\"></div>\n",
       "<script type=\"text/javascript\">\n",
       "  var VEGA_DEBUG = (typeof VEGA_DEBUG == \"undefined\") ? {} : VEGA_DEBUG;\n",
       "  (function(spec, embedOpt){\n",
       "    let outputDiv = document.currentScript.previousElementSibling;\n",
       "    if (outputDiv.id !== \"altair-viz-d09af60bd6344262a6b3c497f86ff49a\") {\n",
       "      outputDiv = document.getElementById(\"altair-viz-d09af60bd6344262a6b3c497f86ff49a\");\n",
       "    }\n",
       "    const paths = {\n",
       "      \"vega\": \"https://cdn.jsdelivr.net/npm/vega@5?noext\",\n",
       "      \"vega-lib\": \"https://cdn.jsdelivr.net/npm/vega-lib?noext\",\n",
       "      \"vega-lite\": \"https://cdn.jsdelivr.net/npm/vega-lite@5.8.0?noext\",\n",
       "      \"vega-embed\": \"https://cdn.jsdelivr.net/npm/vega-embed@6?noext\",\n",
       "    };\n",
       "\n",
       "    function maybeLoadScript(lib, version) {\n",
       "      var key = `${lib.replace(\"-\", \"\")}_version`;\n",
       "      return (VEGA_DEBUG[key] == version) ?\n",
       "        Promise.resolve(paths[lib]) :\n",
       "        new Promise(function(resolve, reject) {\n",
       "          var s = document.createElement('script');\n",
       "          document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
       "          s.async = true;\n",
       "          s.onload = () => {\n",
       "            VEGA_DEBUG[key] = version;\n",
       "            return resolve(paths[lib]);\n",
       "          };\n",
       "          s.onerror = () => reject(`Error loading script: ${paths[lib]}`);\n",
       "          s.src = paths[lib];\n",
       "        });\n",
       "    }\n",
       "\n",
       "    function showError(err) {\n",
       "      outputDiv.innerHTML = `<div class=\"error\" style=\"color:red;\">${err}</div>`;\n",
       "      throw err;\n",
       "    }\n",
       "\n",
       "    function displayChart(vegaEmbed) {\n",
       "      vegaEmbed(outputDiv, spec, embedOpt)\n",
       "        .catch(err => showError(`Javascript Error: ${err.message}<br>This usually means there's a typo in your chart specification. See the javascript console for the full traceback.`));\n",
       "    }\n",
       "\n",
       "    if(typeof define === \"function\" && define.amd) {\n",
       "      requirejs.config({paths});\n",
       "      require([\"vega-embed\"], displayChart, err => showError(`Error loading script: ${err.message}`));\n",
       "    } else {\n",
       "      maybeLoadScript(\"vega\", \"5\")\n",
       "        .then(() => maybeLoadScript(\"vega-lite\", \"5.8.0\"))\n",
       "        .then(() => maybeLoadScript(\"vega-embed\", \"6\"))\n",
       "        .catch(showError)\n",
       "        .then(() => displayChart(vegaEmbed));\n",
       "    }\n",
       "  })({\"config\": {\"view\": {\"continuousWidth\": 300, \"continuousHeight\": 300, \"step\": 15}}, \"layer\": [{\"mark\": {\"type\": \"bar\"}, \"encoding\": {\"color\": {\"field\": \"comparison_name\", \"legend\": null, \"type\": \"nominal\"}, \"tooltip\": [{\"field\": \"comparison_name\", \"type\": \"nominal\"}, {\"field\": \"label_for_charts\", \"type\": \"nominal\"}, {\"field\": \"sql_condition\", \"type\": \"nominal\"}, {\"field\": \"m_probability\", \"type\": \"quantitative\"}, {\"field\": \"u_probability\", \"type\": \"quantitative\"}, {\"field\": \"bayes_factor\", \"type\": \"quantitative\"}, {\"field\": \"log2_bayes_factor\", \"type\": \"quantitative\"}], \"x\": {\"field\": \"log2_bayes_factor\", \"scale\": {\"domain\": [-10, 10]}, \"title\": \"Comparison level match weight = log2(m/u)\", \"type\": \"quantitative\"}, \"y\": {\"field\": \"cl_id\", \"sort\": \"-x\", \"title\": \"Comparison level\", \"type\": \"nominal\"}}}, {\"mark\": {\"type\": \"text\", \"align\": \"right\", \"dx\": 0}, \"encoding\": {\"text\": {\"field\": \"comparison_name\", \"type\": \"nominal\"}, \"tooltip\": [{\"field\": \"comparison_name\", \"type\": \"nominal\"}, {\"field\": \"label_for_charts\", \"type\": \"nominal\"}, {\"field\": \"sql_condition\", \"type\": \"nominal\"}, {\"field\": \"m_probability\", \"type\": \"quantitative\"}, {\"field\": \"u_probability\", \"type\": \"quantitative\"}, {\"field\": \"bayes_factor\", \"type\": \"quantitative\"}, {\"field\": \"log2_bayes_factor\", \"type\": \"quantitative\"}], \"x\": {\"field\": \"log2_bayes_factor\", \"scale\": {\"domain\": [-10, 10]}, \"title\": \"Comparison level match weight = log2(m/u)\", \"type\": \"quantitative\"}, \"y\": {\"field\": \"cl_id\", \"sort\": \"-x\", \"title\": \"Comparison level\", \"type\": \"nominal\"}}}], \"data\": {\"name\": \"data-3901c03d78701611834aa82ab7374cce\"}, \"resolve\": {\"axis\": {\"x\": \"shared\", \"y\": \"shared\"}}, \"title\": \"New Chart - WOO!\", \"$schema\": \"https://vega.github.io/schema/vega-lite/v5.8.0.json\", \"datasets\": {\"data-3901c03d78701611834aa82ab7374cce\": [{\"comparison_name\": \"first_name\", \"sql_condition\": \"\\\"first_name_l\\\" = \\\"first_name_r\\\"\", \"label_for_charts\": \"Exact match first_name\", \"m_probability\": 0.5018941916173814, \"u_probability\": 0.0057935713975033705, \"bayes_factor\": 86.62949969575988, \"log2_bayes_factor\": 6.436786480320881, \"comparison_vector_value\": 4, \"cl_id\": \"first_name_4\"}, {\"comparison_name\": \"first_name\", \"sql_condition\": \"damerau_levenshtein(\\\"first_name_l\\\", \\\"first_name_r\\\") <= 1\", \"label_for_charts\": \"Damerau_levenshtein <= 1\", \"m_probability\": 0.19595791797531015, \"u_probability\": 0.00236614327345483, \"bayes_factor\": 82.81743551783742, \"log2_bayes_factor\": 6.371862624533329, \"comparison_vector_value\": 3, \"cl_id\": \"first_name_3\"}, {\"comparison_name\": \"first_name\", \"sql_condition\": \"jaro_winkler_similarity(\\\"first_name_l\\\", \\\"first_name_r\\\") >= 0.9\", \"label_for_charts\": \"Jaro_winkler_similarity >= 0.9\", \"m_probability\": 0.045985303626033085, \"u_probability\": 0.001296159366708712, \"bayes_factor\": 35.47812468678278, \"log2_bayes_factor\": 5.148857848140163, \"comparison_vector_value\": 2, \"cl_id\": \"first_name_2\"}, {\"comparison_name\": \"first_name\", \"sql_condition\": \"jaro_winkler_similarity(\\\"first_name_l\\\", \\\"first_name_r\\\") >= 0.8\", \"label_for_charts\": \"Jaro_winkler_similarity >= 0.8\", \"m_probability\": 0.06396730257493154, \"u_probability\": 0.005677583982137938, \"bayes_factor\": 11.266641370022352, \"log2_bayes_factor\": 3.493985601438375, \"comparison_vector_value\": 1, \"cl_id\": \"first_name_1\"}, {\"comparison_name\": \"first_name\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.19219528420634394, \"u_probability\": 0.9848665419801952, \"bayes_factor\": 0.19514855669673956, \"log2_bayes_factor\": -2.357355302129234, \"comparison_vector_value\": 0, \"cl_id\": \"first_name_0\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"\\\"surname_l\\\" = \\\"surname_r\\\"\", \"label_for_charts\": \"Exact match surname\", \"m_probability\": 0.5527050424941531, \"u_probability\": 0.004889975550122249, \"bayes_factor\": 113.02818119005431, \"log2_bayes_factor\": 6.820538712806792, \"comparison_vector_value\": 4, \"cl_id\": \"surname_4\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"damerau_levenshtein(\\\"surname_l\\\", \\\"surname_r\\\") <= 1\", \"label_for_charts\": \"Damerau_levenshtein <= 1\", \"m_probability\": 0.22212752320956386, \"u_probability\": 0.0027554624131641246, \"bayes_factor\": 80.61351958508214, \"log2_bayes_factor\": 6.332949906378981, \"comparison_vector_value\": 3, \"cl_id\": \"surname_3\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"jaro_winkler_similarity(\\\"surname_l\\\", \\\"surname_r\\\") >= 0.9\", \"label_for_charts\": \"Jaro_winkler_similarity >= 0.9\", \"m_probability\": 0.0490149338194711, \"u_probability\": 0.0010090425738347498, \"bayes_factor\": 48.57568460485815, \"log2_bayes_factor\": 5.602162423566203, \"comparison_vector_value\": 2, \"cl_id\": \"surname_2\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"jaro_winkler_similarity(\\\"surname_l\\\", \\\"surname_r\\\") >= 0.8\", \"label_for_charts\": \"Jaro_winkler_similarity >= 0.8\", \"m_probability\": 0.05001678986356945, \"u_probability\": 0.003710768991942586, \"bayes_factor\": 13.478820689774516, \"log2_bayes_factor\": 3.752622370380284, \"comparison_vector_value\": 1, \"cl_id\": \"surname_1\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.1261357106132424, \"u_probability\": 0.9876347504709363, \"bayes_factor\": 0.1277149376863226, \"log2_bayes_factor\": -2.969000820703079, \"comparison_vector_value\": 0, \"cl_id\": \"surname_0\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"\\\"dob_l\\\" = \\\"dob_r\\\"\", \"label_for_charts\": \"Exact match\", \"m_probability\": 0.41383785481447766, \"u_probability\": 0.0017477477477477479, \"bayes_factor\": 236.78351486807742, \"log2_bayes_factor\": 7.887424832202931, \"comparison_vector_value\": 5, \"cl_id\": \"dob_5\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"damerau_levenshtein(\\\"dob_l\\\", \\\"dob_r\\\") <= 1\", \"label_for_charts\": \"Damerau_levenshtein <= 1\", \"m_probability\": 0.10806341031654734, \"u_probability\": 0.0016436436436436436, \"bayes_factor\": 65.74625268345359, \"log2_bayes_factor\": 6.038836762842662, \"comparison_vector_value\": 4, \"cl_id\": \"dob_4\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"\\n            abs(date_diff('month',\\n                strptime(\\\"dob_l\\\", '%Y-%m-%d'),\\n                strptime(\\\"dob_r\\\", '%Y-%m-%d'))\\n                ) <= 1\\n        \", \"label_for_charts\": \"Within 1 month\", \"m_probability\": 0.11300938544779224, \"u_probability\": 0.003833833833833834, \"bayes_factor\": 29.476860590690453, \"log2_bayes_factor\": 4.881510974428093, \"comparison_vector_value\": 3, \"cl_id\": \"dob_3\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"\\n            abs(date_diff('year',\\n                strptime(\\\"dob_l\\\", '%Y-%m-%d'),\\n                strptime(\\\"dob_r\\\", '%Y-%m-%d'))\\n                ) <= 1\\n        \", \"label_for_charts\": \"Within 1 year\", \"m_probability\": 0.17200656922328977, \"u_probability\": 0.05062662662662663, \"bayes_factor\": 3.397551460259144, \"log2_bayes_factor\": 1.7644954026183992, \"comparison_vector_value\": 2, \"cl_id\": \"dob_2\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"\\n            abs(date_diff('year',\\n                strptime(\\\"dob_l\\\", '%Y-%m-%d'),\\n                strptime(\\\"dob_r\\\", '%Y-%m-%d'))\\n                ) <= 10\\n        \", \"label_for_charts\": \"Within 10 years\", \"m_probability\": 0.19035523041792068, \"u_probability\": 0.3037037037037037, \"bayes_factor\": 0.6267794172297388, \"log2_bayes_factor\": -0.6739702908716182, \"comparison_vector_value\": 1, \"cl_id\": \"dob_1\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.002727549779972325, \"u_probability\": 0.6384444444444445, \"bayes_factor\": 0.004272180302776005, \"log2_bayes_factor\": -7.870811748958801, \"comparison_vector_value\": 0, \"cl_id\": \"dob_0\"}, {\"comparison_name\": \"city\", \"sql_condition\": \"\\\"city_l\\\" = \\\"city_r\\\"\", \"label_for_charts\": \"Exact match\", \"m_probability\": 0.6013808934279701, \"u_probability\": 0.0551475711801453, \"bayes_factor\": 10.904938885948333, \"log2_bayes_factor\": 3.4469097796586596, \"comparison_vector_value\": 1, \"cl_id\": \"city_1\"}, {\"comparison_name\": \"city\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.3986191065720299, \"u_probability\": 0.9448524288198547, \"bayes_factor\": 0.42188504195296994, \"log2_bayes_factor\": -1.2450781575619725, \"comparison_vector_value\": 0, \"cl_id\": \"city_0\"}, {\"comparison_name\": \"email\", \"sql_condition\": \"\\\"email_l\\\" = \\\"email_r\\\"\", \"label_for_charts\": \"Exact match\", \"m_probability\": 0.5914840252879943, \"u_probability\": 0.0021938713143283602, \"bayes_factor\": 269.6074384240141, \"log2_bayes_factor\": 8.07471649055784, \"comparison_vector_value\": 2, \"cl_id\": \"email_2\"}, {\"comparison_name\": \"email\", \"sql_condition\": \"levenshtein(\\\"email_l\\\", \\\"email_r\\\") <= 2\", \"label_for_charts\": \"Levenshtein <= 2\", \"m_probability\": 0.3019669634613132, \"u_probability\": 0.0013542812658830492, \"bayes_factor\": 222.9721189153553, \"log2_bayes_factor\": 7.800719512398763, \"comparison_vector_value\": 1, \"cl_id\": \"email_1\"}, {\"comparison_name\": \"email\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.10654901125069259, \"u_probability\": 0.9964518474197885, \"bayes_factor\": 0.10692840956298139, \"log2_bayes_factor\": -3.225282884575804, \"comparison_vector_value\": 0, \"cl_id\": \"email_0\"}]}}, {\"mode\": \"vega-lite\"});\n",
       "</script>"
      ],
      "text/plain": [
       "alt.LayerChart(...)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create base chart with shared data and encodings (mark type not specified)\n",
    "base = alt.Chart(df).encode(\n",
    "    y=alt.Y(\"cl_id\",\n",
    "            sort=\"-x\",\n",
    "            title=\"Comparison level\"\n",
    "            ),\n",
    "    x=alt.X(\"log2_bayes_factor\",\n",
    "            title=\"Comparison level match weight = log2(m/u)\",\n",
    "            scale=alt.Scale(domain=[-10, 10])\n",
    "            ),\n",
    "    tooltip=[\n",
    "        \"comparison_name\",\n",
    "        \"label_for_charts\",\n",
    "        \"sql_condition\",\n",
    "        \"m_probability\",\n",
    "        \"u_probability\",\n",
    "        \"bayes_factor\",\n",
    "        \"log2_bayes_factor\"\n",
    "    ]\n",
    ")\n",
    "\n",
    "# Build bar chart from base (color legend made redundant by text labels)\n",
    "bar = base.mark_bar().encode(\n",
    "    color=alt.Color(\"comparison_name\", legend=None)\n",
    ")\n",
    "\n",
    "# Build text layer from base\n",
    "text = base.mark_text(dx=0, align=\"right\").encode(\n",
    "    text=\"comparison_name\"\n",
    ")\n",
    "\n",
    "# Final layered chart\n",
    "chart = bar + text\n",
    "\n",
    "# Add global config\n",
    "chart.resolve_axis(\n",
    "    y=\"shared\",\n",
    "    x=\"shared\"\n",
    ").properties(\n",
    "    title=\"New Chart - WOO!\"\n",
    ").configure_view(\n",
    "    step=15\n",
    ")\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes things go wrong in Altair and it's not clear why or how to fix it. If the docs and Stack Overflow don't have a solution, the answer is usually that Altair is making decisions under the hood about the Vega-Lite schema that are out of your control.\n",
    "\n",
    "In this example, the sorting of the y-axis is broken when layering charts. If we show `bar` and `text` side-by-side, you can see they work as expected, but the sorting is broken in the layering process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "<style>\n",
       "  #altair-viz-5af5714a540a4cee9e489f6f7c421811.vega-embed {\n",
       "    width: 100%;\n",
       "    display: flex;\n",
       "  }\n",
       "\n",
       "  #altair-viz-5af5714a540a4cee9e489f6f7c421811.vega-embed details,\n",
       "  #altair-viz-5af5714a540a4cee9e489f6f7c421811.vega-embed details summary {\n",
       "    position: relative;\n",
       "  }\n",
       "</style>\n",
       "<div id=\"altair-viz-5af5714a540a4cee9e489f6f7c421811\"></div>\n",
       "<script type=\"text/javascript\">\n",
       "  var VEGA_DEBUG = (typeof VEGA_DEBUG == \"undefined\") ? {} : VEGA_DEBUG;\n",
       "  (function(spec, embedOpt){\n",
       "    let outputDiv = document.currentScript.previousElementSibling;\n",
       "    if (outputDiv.id !== \"altair-viz-5af5714a540a4cee9e489f6f7c421811\") {\n",
       "      outputDiv = document.getElementById(\"altair-viz-5af5714a540a4cee9e489f6f7c421811\");\n",
       "    }\n",
       "    const paths = {\n",
       "      \"vega\": \"https://cdn.jsdelivr.net/npm/vega@5?noext\",\n",
       "      \"vega-lib\": \"https://cdn.jsdelivr.net/npm/vega-lib?noext\",\n",
       "      \"vega-lite\": \"https://cdn.jsdelivr.net/npm/vega-lite@5.8.0?noext\",\n",
       "      \"vega-embed\": \"https://cdn.jsdelivr.net/npm/vega-embed@6?noext\",\n",
       "    };\n",
       "\n",
       "    function maybeLoadScript(lib, version) {\n",
       "      var key = `${lib.replace(\"-\", \"\")}_version`;\n",
       "      return (VEGA_DEBUG[key] == version) ?\n",
       "        Promise.resolve(paths[lib]) :\n",
       "        new Promise(function(resolve, reject) {\n",
       "          var s = document.createElement('script');\n",
       "          document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
       "          s.async = true;\n",
       "          s.onload = () => {\n",
       "            VEGA_DEBUG[key] = version;\n",
       "            return resolve(paths[lib]);\n",
       "          };\n",
       "          s.onerror = () => reject(`Error loading script: ${paths[lib]}`);\n",
       "          s.src = paths[lib];\n",
       "        });\n",
       "    }\n",
       "\n",
       "    function showError(err) {\n",
       "      outputDiv.innerHTML = `<div class=\"error\" style=\"color:red;\">${err}</div>`;\n",
       "      throw err;\n",
       "    }\n",
       "\n",
       "    function displayChart(vegaEmbed) {\n",
       "      vegaEmbed(outputDiv, spec, embedOpt)\n",
       "        .catch(err => showError(`Javascript Error: ${err.message}<br>This usually means there's a typo in your chart specification. See the javascript console for the full traceback.`));\n",
       "    }\n",
       "\n",
       "    if(typeof define === \"function\" && define.amd) {\n",
       "      requirejs.config({paths});\n",
       "      require([\"vega-embed\"], displayChart, err => showError(`Error loading script: ${err.message}`));\n",
       "    } else {\n",
       "      maybeLoadScript(\"vega\", \"5\")\n",
       "        .then(() => maybeLoadScript(\"vega-lite\", \"5.8.0\"))\n",
       "        .then(() => maybeLoadScript(\"vega-embed\", \"6\"))\n",
       "        .catch(showError)\n",
       "        .then(() => displayChart(vegaEmbed));\n",
       "    }\n",
       "  })({\"config\": {\"view\": {\"continuousWidth\": 300, \"continuousHeight\": 300}}, \"hconcat\": [{\"mark\": {\"type\": \"bar\"}, \"encoding\": {\"color\": {\"field\": \"comparison_name\", \"legend\": null, \"type\": \"nominal\"}, \"tooltip\": [{\"field\": \"comparison_name\", \"type\": \"nominal\"}, {\"field\": \"label_for_charts\", \"type\": \"nominal\"}, {\"field\": \"sql_condition\", \"type\": \"nominal\"}, {\"field\": \"m_probability\", \"type\": \"quantitative\"}, {\"field\": \"u_probability\", \"type\": \"quantitative\"}, {\"field\": \"bayes_factor\", \"type\": \"quantitative\"}, {\"field\": \"log2_bayes_factor\", \"type\": \"quantitative\"}], \"x\": {\"field\": \"log2_bayes_factor\", \"scale\": {\"domain\": [-10, 10]}, \"title\": \"Comparison level match weight = log2(m/u)\", \"type\": \"quantitative\"}, \"y\": {\"field\": \"cl_id\", \"sort\": \"-x\", \"title\": \"Comparison level\", \"type\": \"nominal\"}}}, {\"mark\": {\"type\": \"text\", \"align\": \"right\", \"dx\": 0}, \"encoding\": {\"text\": {\"field\": \"comparison_name\", \"type\": \"nominal\"}, \"tooltip\": [{\"field\": \"comparison_name\", \"type\": \"nominal\"}, {\"field\": \"label_for_charts\", \"type\": \"nominal\"}, {\"field\": \"sql_condition\", \"type\": \"nominal\"}, {\"field\": \"m_probability\", \"type\": \"quantitative\"}, {\"field\": \"u_probability\", \"type\": \"quantitative\"}, {\"field\": \"bayes_factor\", \"type\": \"quantitative\"}, {\"field\": \"log2_bayes_factor\", \"type\": \"quantitative\"}], \"x\": {\"field\": \"log2_bayes_factor\", \"scale\": {\"domain\": [-10, 10]}, \"title\": \"Comparison level match weight = log2(m/u)\", \"type\": \"quantitative\"}, \"y\": {\"field\": \"cl_id\", \"sort\": \"-x\", \"title\": \"Comparison level\", \"type\": \"nominal\"}}}], \"data\": {\"name\": \"data-3901c03d78701611834aa82ab7374cce\"}, \"$schema\": \"https://vega.github.io/schema/vega-lite/v5.8.0.json\", \"datasets\": {\"data-3901c03d78701611834aa82ab7374cce\": [{\"comparison_name\": \"first_name\", \"sql_condition\": \"\\\"first_name_l\\\" = \\\"first_name_r\\\"\", \"label_for_charts\": \"Exact match first_name\", \"m_probability\": 0.5018941916173814, \"u_probability\": 0.0057935713975033705, \"bayes_factor\": 86.62949969575988, \"log2_bayes_factor\": 6.436786480320881, \"comparison_vector_value\": 4, \"cl_id\": \"first_name_4\"}, {\"comparison_name\": \"first_name\", \"sql_condition\": \"damerau_levenshtein(\\\"first_name_l\\\", \\\"first_name_r\\\") <= 1\", \"label_for_charts\": \"Damerau_levenshtein <= 1\", \"m_probability\": 0.19595791797531015, \"u_probability\": 0.00236614327345483, \"bayes_factor\": 82.81743551783742, \"log2_bayes_factor\": 6.371862624533329, \"comparison_vector_value\": 3, \"cl_id\": \"first_name_3\"}, {\"comparison_name\": \"first_name\", \"sql_condition\": \"jaro_winkler_similarity(\\\"first_name_l\\\", \\\"first_name_r\\\") >= 0.9\", \"label_for_charts\": \"Jaro_winkler_similarity >= 0.9\", \"m_probability\": 0.045985303626033085, \"u_probability\": 0.001296159366708712, \"bayes_factor\": 35.47812468678278, \"log2_bayes_factor\": 5.148857848140163, \"comparison_vector_value\": 2, \"cl_id\": \"first_name_2\"}, {\"comparison_name\": \"first_name\", \"sql_condition\": \"jaro_winkler_similarity(\\\"first_name_l\\\", \\\"first_name_r\\\") >= 0.8\", \"label_for_charts\": \"Jaro_winkler_similarity >= 0.8\", \"m_probability\": 0.06396730257493154, \"u_probability\": 0.005677583982137938, \"bayes_factor\": 11.266641370022352, \"log2_bayes_factor\": 3.493985601438375, \"comparison_vector_value\": 1, \"cl_id\": \"first_name_1\"}, {\"comparison_name\": \"first_name\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.19219528420634394, \"u_probability\": 0.9848665419801952, \"bayes_factor\": 0.19514855669673956, \"log2_bayes_factor\": -2.357355302129234, \"comparison_vector_value\": 0, \"cl_id\": \"first_name_0\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"\\\"surname_l\\\" = \\\"surname_r\\\"\", \"label_for_charts\": \"Exact match surname\", \"m_probability\": 0.5527050424941531, \"u_probability\": 0.004889975550122249, \"bayes_factor\": 113.02818119005431, \"log2_bayes_factor\": 6.820538712806792, \"comparison_vector_value\": 4, \"cl_id\": \"surname_4\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"damerau_levenshtein(\\\"surname_l\\\", \\\"surname_r\\\") <= 1\", \"label_for_charts\": \"Damerau_levenshtein <= 1\", \"m_probability\": 0.22212752320956386, \"u_probability\": 0.0027554624131641246, \"bayes_factor\": 80.61351958508214, \"log2_bayes_factor\": 6.332949906378981, \"comparison_vector_value\": 3, \"cl_id\": \"surname_3\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"jaro_winkler_similarity(\\\"surname_l\\\", \\\"surname_r\\\") >= 0.9\", \"label_for_charts\": \"Jaro_winkler_similarity >= 0.9\", \"m_probability\": 0.0490149338194711, \"u_probability\": 0.0010090425738347498, \"bayes_factor\": 48.57568460485815, \"log2_bayes_factor\": 5.602162423566203, \"comparison_vector_value\": 2, \"cl_id\": \"surname_2\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"jaro_winkler_similarity(\\\"surname_l\\\", \\\"surname_r\\\") >= 0.8\", \"label_for_charts\": \"Jaro_winkler_similarity >= 0.8\", \"m_probability\": 0.05001678986356945, \"u_probability\": 0.003710768991942586, \"bayes_factor\": 13.478820689774516, \"log2_bayes_factor\": 3.752622370380284, \"comparison_vector_value\": 1, \"cl_id\": \"surname_1\"}, {\"comparison_name\": \"surname\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.1261357106132424, \"u_probability\": 0.9876347504709363, \"bayes_factor\": 0.1277149376863226, \"log2_bayes_factor\": -2.969000820703079, \"comparison_vector_value\": 0, \"cl_id\": \"surname_0\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"\\\"dob_l\\\" = \\\"dob_r\\\"\", \"label_for_charts\": \"Exact match\", \"m_probability\": 0.41383785481447766, \"u_probability\": 0.0017477477477477479, \"bayes_factor\": 236.78351486807742, \"log2_bayes_factor\": 7.887424832202931, \"comparison_vector_value\": 5, \"cl_id\": \"dob_5\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"damerau_levenshtein(\\\"dob_l\\\", \\\"dob_r\\\") <= 1\", \"label_for_charts\": \"Damerau_levenshtein <= 1\", \"m_probability\": 0.10806341031654734, \"u_probability\": 0.0016436436436436436, \"bayes_factor\": 65.74625268345359, \"log2_bayes_factor\": 6.038836762842662, \"comparison_vector_value\": 4, \"cl_id\": \"dob_4\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"\\n            abs(date_diff('month',\\n                strptime(\\\"dob_l\\\", '%Y-%m-%d'),\\n                strptime(\\\"dob_r\\\", '%Y-%m-%d'))\\n                ) <= 1\\n        \", \"label_for_charts\": \"Within 1 month\", \"m_probability\": 0.11300938544779224, \"u_probability\": 0.003833833833833834, \"bayes_factor\": 29.476860590690453, \"log2_bayes_factor\": 4.881510974428093, \"comparison_vector_value\": 3, \"cl_id\": \"dob_3\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"\\n            abs(date_diff('year',\\n                strptime(\\\"dob_l\\\", '%Y-%m-%d'),\\n                strptime(\\\"dob_r\\\", '%Y-%m-%d'))\\n                ) <= 1\\n        \", \"label_for_charts\": \"Within 1 year\", \"m_probability\": 0.17200656922328977, \"u_probability\": 0.05062662662662663, \"bayes_factor\": 3.397551460259144, \"log2_bayes_factor\": 1.7644954026183992, \"comparison_vector_value\": 2, \"cl_id\": \"dob_2\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"\\n            abs(date_diff('year',\\n                strptime(\\\"dob_l\\\", '%Y-%m-%d'),\\n                strptime(\\\"dob_r\\\", '%Y-%m-%d'))\\n                ) <= 10\\n        \", \"label_for_charts\": \"Within 10 years\", \"m_probability\": 0.19035523041792068, \"u_probability\": 0.3037037037037037, \"bayes_factor\": 0.6267794172297388, \"log2_bayes_factor\": -0.6739702908716182, \"comparison_vector_value\": 1, \"cl_id\": \"dob_1\"}, {\"comparison_name\": \"dob\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.002727549779972325, \"u_probability\": 0.6384444444444445, \"bayes_factor\": 0.004272180302776005, \"log2_bayes_factor\": -7.870811748958801, \"comparison_vector_value\": 0, \"cl_id\": \"dob_0\"}, {\"comparison_name\": \"city\", \"sql_condition\": \"\\\"city_l\\\" = \\\"city_r\\\"\", \"label_for_charts\": \"Exact match\", \"m_probability\": 0.6013808934279701, \"u_probability\": 0.0551475711801453, \"bayes_factor\": 10.904938885948333, \"log2_bayes_factor\": 3.4469097796586596, \"comparison_vector_value\": 1, \"cl_id\": \"city_1\"}, {\"comparison_name\": \"city\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.3986191065720299, \"u_probability\": 0.9448524288198547, \"bayes_factor\": 0.42188504195296994, \"log2_bayes_factor\": -1.2450781575619725, \"comparison_vector_value\": 0, \"cl_id\": \"city_0\"}, {\"comparison_name\": \"email\", \"sql_condition\": \"\\\"email_l\\\" = \\\"email_r\\\"\", \"label_for_charts\": \"Exact match\", \"m_probability\": 0.5914840252879943, \"u_probability\": 0.0021938713143283602, \"bayes_factor\": 269.6074384240141, \"log2_bayes_factor\": 8.07471649055784, \"comparison_vector_value\": 2, \"cl_id\": \"email_2\"}, {\"comparison_name\": \"email\", \"sql_condition\": \"levenshtein(\\\"email_l\\\", \\\"email_r\\\") <= 2\", \"label_for_charts\": \"Levenshtein <= 2\", \"m_probability\": 0.3019669634613132, \"u_probability\": 0.0013542812658830492, \"bayes_factor\": 222.9721189153553, \"log2_bayes_factor\": 7.800719512398763, \"comparison_vector_value\": 1, \"cl_id\": \"email_1\"}, {\"comparison_name\": \"email\", \"sql_condition\": \"ELSE\", \"label_for_charts\": \"All other comparisons\", \"m_probability\": 0.10654901125069259, \"u_probability\": 0.9964518474197885, \"bayes_factor\": 0.10692840956298139, \"log2_bayes_factor\": -3.225282884575804, \"comparison_vector_value\": 0, \"cl_id\": \"email_0\"}]}}, {\"mode\": \"vega-lite\"});\n",
       "</script>"
      ],
      "text/plain": [
       "alt.HConcatChart(...)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bar | text"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we get to this stage (or whenever you're comfortable), we can switch to Vega-Lite by exporting the JSON from our `chart` object, or opening the chart in the Vega-Lite editor.\n",
    "\n",
    "```py\n",
    "chart.to_json()\n",
    "```"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "??? note \"Chart JSON\"\n",
    "      ```json\n",
    "        {\n",
    "        \"$schema\": \"https://vega.github.io/schema/vega-lite/v5.8.0.json\",\n",
    "        \"config\": {\n",
    "          \"view\": {\n",
    "            \"continuousHeight\": 300,\n",
    "            \"continuousWidth\": 300\n",
    "          }\n",
    "        },\n",
    "        \"data\": {\n",
    "          \"name\": \"data-3901c03d78701611834aa82ab7374cce\"\n",
    "        },\n",
    "        \"datasets\": {\n",
    "          \"data-3901c03d78701611834aa82ab7374cce\": [\n",
    "            {\n",
    "              \"bayes_factor\": 86.62949969575988,\n",
    "              \"cl_id\": \"first_name_4\",\n",
    "              \"comparison_name\": \"first_name\",\n",
    "              \"comparison_vector_value\": 4,\n",
    "              \"label_for_charts\": \"Exact match first_name\",\n",
    "              \"log2_bayes_factor\": 6.436786480320881,\n",
    "              \"m_probability\": 0.5018941916173814,\n",
    "              \"sql_condition\": \"\\\"first_name_l\\\" = \\\"first_name_r\\\"\",\n",
    "              \"u_probability\": 0.0057935713975033705\n",
    "            },\n",
    "            {\n",
    "              \"bayes_factor\": 82.81743551783742,\n",
    "              \"cl_id\": \"first_name_3\",\n",
    "              \"comparison_name\": \"first_name\",\n",
    "              \"comparison_vector_value\": 3,\n",
    "              \"label_for_charts\": \"Damerau_levenshtein <= 1\",\n",
    "              \"log2_bayes_factor\": 6.371862624533329,\n",
    "              \"m_probability\": 0.19595791797531015,\n",
    "              \"sql_condition\": \"damerau_levenshtein(\\\"first_name_l\\\", \\\"first_name_r\\\") <= 1\",\n",
    "              \"u_probability\": 0.00236614327345483\n",
    "            },\n",
    "            {\n",
    "              \"bayes_factor\": 35.47812468678278,\n",
    "              \"cl_id\": \"first_name_2\",\n",
    "              \"comparison_name\": \"first_name\",\n",
    "              \"comparison_vector_value\": 2,\n",
    "              \"label_for_charts\": \"Jaro_winkler_similarity >= 0.9\",\n",
    "              \"log2_bayes_factor\": 5.148857848140163,\n",
    "              \"m_probability\": 0.045985303626033085,\n",
    "              \"sql_condition\": \"jaro_winkler_similarity(\\\"first_name_l\\\", \\\"first_name_r\\\") >= 0.9\",\n",
    "              \"u_probability\": 0.001296159366708712\n",
    "            },\n",
    "            {\n",
    "              \"bayes_factor\": 11.266641370022352,\n",
    "              \"cl_id\": \"first_name_1\",\n",
    "              \"comparison_name\": \"first_name\",\n",
    "              \"comparison_vector_value\": 1,\n",
    "              \"label_for_charts\": \"Jaro_winkler_similarity >= 0.8\",\n",
    "              \"log2_bayes_factor\": 3.493985601438375,\n",
    "              \"m_probability\": 0.06396730257493154,\n",
    "              \"sql_condition\": \"jaro_winkler_similarity(\\\"first_name_l\\\", \\\"first_name_r\\\") >= 0.8\",\n",
    "              \"u_probability\": 0.005677583982137938\n",
    "            },\n",
    "            {\n",
    "              \"bayes_factor\": 0.19514855669673956,\n",
    "              \"cl_id\": \"first_name_0\",\n",
    "              \"comparison_name\": \"first_name\",\n",
    "              \"comparison_vector_value\": 0,\n",
    "              \"label_for_charts\": \"All other comparisons\",\n",
    "              \"log2_bayes_factor\": -2.357355302129234,\n",
    "              \"m_probability\": 0.19219528420634394,\n",
    "              \"sql_condition\": \"ELSE\",\n",
    "              \"u_probability\": 0.9848665419801952\n",
    "            },\n",
    "            {\n",
    "              \"bayes_factor\": 113.02818119005431,\n",
    "              \"cl_id\": \"surname_4\",\n",
    "              \"comparison_name\": \"surname\",\n",
    "              \"comparison_vector_value\": 4,\n",
    "              \"label_for_charts\": \"Exact match surname\",\n",
    "              \"log2_bayes_factor\": 6.820538712806792,\n",
    "              \"m_probability\": 0.5527050424941531,\n",
    "              \"sql_condition\": \"\\\"surname_l\\\" = \\\"surname_r\\\"\",\n",
    "              \"u_probability\": 0.004889975550122249\n",
    "            },\n",
    "            {\n",
    "              \"bayes_factor\": 80.61351958508214,\n",
    "              \"cl_id\": \"surname_3\",\n",
    "              \"comparison_name\": \"surname\",\n",
    "              \"comparison_vector_value\": 3,\n",
    "              \"label_for_charts\": \"Damerau_levenshtein <= 1\",\n",
    "              \"log2_bayes_factor\": 6.332949906378981,\n",
    "              \"m_probability\": 0.22212752320956386,\n",
    "              \"sql_condition\": \"damerau_levenshtein(\\\"surname_l\\\", \\\"surname_r\\\") <= 1\",\n",
    "              \"u_probability\": 0.0027554624131641246\n",
    "            },\n",
    "            {\n",
    "              \"bayes_factor\": 48.57568460485815,\n",
    "              \"cl_id\": \"surname_2\",\n",
    "              \"comparison_name\": \"surname\",\n",
    "              \"comparison_vector_value\": 2,\n",
    "              \"label_for_charts\": \"Jaro_winkler_similarity >= 0.9\",\n",
    "              \"log2_bayes_factor\": 5.602162423566203,\n",
    "              \"m_probability\": 0.0490149338194711,\n",
    "              \"sql_condition\": \"jaro_winkler_similarity(\\\"surname_l\\\", \\\"surname_r\\\") >= 0.9\",\n",
    "              \"u_probability\": 0.0010090425738347498\n",
    "            },\n",
    "            {\n",
    "              \"bayes_factor\": 13.478820689774516,\n",
    "              \"cl_id\": \"surname_1\",\n",
    "              \"comparison_name\": \"surname\",\n",
    "              \"comparison_vector_value\": 1,\n",
    "              \"label_for_charts\": \"Jaro_winkler_similarity >= 0.8\",\n",
    "              \"log2_bayes_factor\": 3.752622370380284,\n",
    "              \"m_probability\": 0.05001678986356945,\n",
    "              \"sql_condition\": \"jaro_winkler_similarity(\\\"surname_l\\\", \\\"surname_r\\\") >= 0.8\",\n",
    "              \"u_probability\": 0.003710768991942586\n",
    "            },\n",
    "            {\n",
    "              \"bayes_factor\": 0.1277149376863226,\n",
    "              \"cl_id\": \"surname_0\",\n",
    "              \"comparison_name\": \"surname\",\n",
    "              \"comparison_vector_value\": 0,\n",
    "              \"label_for_charts\": \"All other comparisons\",\n",
    "              \"log2_bayes_factor\": -2.969000820703079,\n",
    "              \"m_probability\": 0.1261357106132424,\n",
    "              \"sql_condition\": \"ELSE\",\n",
    "              \"u_probability\": 0.9876347504709363\n",
    "            },\n",
    "            {\n",
    "              \"bayes_factor\": 236.78351486807742,\n",
    "              \"cl_id\": \"dob_5\",\n",
    "              \"comparison_name\": \"dob\",\n",
    "              \"comparison_vector_value\": 5,\n",
    "              \"label_for_charts\": \"Exact match\",\n",
    "              \"log2_bayes_factor\": 7.887424832202931,\n",
    "              \"m_probability\": 0.41383785481447766,\n",
    "              \"sql_condition\": \"\\\"dob_l\\\" = \\\"dob_r\\\"\",\n",
    "              \"u_probability\": 0.0017477477477477479\n",
    "            },\n",
    "            {\n",
    "              \"bayes_factor\": 65.74625268345359,\n",
    "              \"cl_id\": \"dob_4\",\n",
    "              \"comparison_name\": \"dob\",\n",
    "              \"comparison_vector_value\": 4,\n",
    "              \"label_for_charts\": \"Damerau_levenshtein <= 1\",\n",
    "              \"log2_bayes_factor\": 6.038836762842662,\n",
    "              \"m_probability\": 0.10806341031654734,\n",
    "              \"sql_condition\": \"damerau_levenshtein(\\\"dob_l\\\", \\\"dob_r\\\") <= 1\",\n",
    "              \"u_probability\": 0.0016436436436436436\n",
    "            },\n",
    "            {\n",
    "              \"bayes_factor\": 29.476860590690453,\n",
    "              \"cl_id\": \"dob_3\",\n",
    "              \"comparison_name\": \"dob\",\n",
    "              \"comparison_vector_value\": 3,\n",
    "              \"label_for_charts\": \"Within 1 month\",\n",
    "              \"log2_bayes_factor\": 4.881510974428093,\n",
    "              \"m_probability\": 0.11300938544779224,\n",
    "              \"sql_condition\": \"\\n            abs(date_diff('month',\\n                strptime(\\\"dob_l\\\", '%Y-%m-%d'),\\n                strptime(\\\"dob_r\\\", '%Y-%m-%d'))\\n                ) <= 1\\n        \",\n",
    "              \"u_probability\": 0.003833833833833834\n",
    "            },\n",
    "            {\n",
    "              \"bayes_factor\": 3.397551460259144,\n",
    "              \"cl_id\": \"dob_2\",\n",
    "              \"comparison_name\": \"dob\",\n",
    "              \"comparison_vector_value\": 2,\n",
    "              \"label_for_charts\": \"Within 1 year\",\n",
    "              \"log2_bayes_factor\": 1.7644954026183992,\n",
    "              \"m_probability\": 0.17200656922328977,\n",
    "              \"sql_condition\": \"\\n            abs(date_diff('year',\\n                strptime(\\\"dob_l\\\", '%Y-%m-%d'),\\n                strptime(\\\"dob_r\\\", '%Y-%m-%d'))\\n                ) <= 1\\n        \",\n",
    "              \"u_probability\": 0.05062662662662663\n",
    "            },\n",
    "            {\n",
    "              \"bayes_factor\": 0.6267794172297388,\n",
    "              \"cl_id\": \"dob_1\",\n",
    "              \"comparison_name\": \"dob\",\n",
    "              \"comparison_vector_value\": 1,\n",
    "              \"label_for_charts\": \"Within 10 years\",\n",
    "              \"log2_bayes_factor\": -0.6739702908716182,\n",
    "              \"m_probability\": 0.19035523041792068,\n",
    "              \"sql_condition\": \"\\n            abs(date_diff('year',\\n                strptime(\\\"dob_l\\\", '%Y-%m-%d'),\\n                strptime(\\\"dob_r\\\", '%Y-%m-%d'))\\n                ) <= 10\\n        \",\n",
    "              \"u_probability\": 0.3037037037037037\n",
    "            },\n",
    "            {\n",
    "              \"bayes_factor\": 0.004272180302776005,\n",
    "              \"cl_id\": \"dob_0\",\n",
    "              \"comparison_name\": \"dob\",\n",
    "              \"comparison_vector_value\": 0,\n",
    "              \"label_for_charts\": \"All other comparisons\",\n",
    "              \"log2_bayes_factor\": -7.870811748958801,\n",
    "              \"m_probability\": 0.002727549779972325,\n",
    "              \"sql_condition\": \"ELSE\",\n",
    "              \"u_probability\": 0.6384444444444445\n",
    "            },\n",
    "            {\n",
    "              \"bayes_factor\": 10.904938885948333,\n",
    "              \"cl_id\": \"city_1\",\n",
    "              \"comparison_name\": \"city\",\n",
    "              \"comparison_vector_value\": 1,\n",
    "              \"label_for_charts\": \"Exact match\",\n",
    "              \"log2_bayes_factor\": 3.4469097796586596,\n",
    "              \"m_probability\": 0.6013808934279701,\n",
    "              \"sql_condition\": \"\\\"city_l\\\" = \\\"city_r\\\"\",\n",
    "              \"u_probability\": 0.0551475711801453\n",
    "            },\n",
    "            {\n",
    "              \"bayes_factor\": 0.42188504195296994,\n",
    "              \"cl_id\": \"city_0\",\n",
    "              \"comparison_name\": \"city\",\n",
    "              \"comparison_vector_value\": 0,\n",
    "              \"label_for_charts\": \"All other comparisons\",\n",
    "              \"log2_bayes_factor\": -1.2450781575619725,\n",
    "              \"m_probability\": 0.3986191065720299,\n",
    "              \"sql_condition\": \"ELSE\",\n",
    "              \"u_probability\": 0.9448524288198547\n",
    "            },\n",
    "            {\n",
    "              \"bayes_factor\": 269.6074384240141,\n",
    "              \"cl_id\": \"email_2\",\n",
    "              \"comparison_name\": \"email\",\n",
    "              \"comparison_vector_value\": 2,\n",
    "              \"label_for_charts\": \"Exact match\",\n",
    "              \"log2_bayes_factor\": 8.07471649055784,\n",
    "              \"m_probability\": 0.5914840252879943,\n",
    "              \"sql_condition\": \"\\\"email_l\\\" = \\\"email_r\\\"\",\n",
    "              \"u_probability\": 0.0021938713143283602\n",
    "            },\n",
    "            {\n",
    "              \"bayes_factor\": 222.9721189153553,\n",
    "              \"cl_id\": \"email_1\",\n",
    "              \"comparison_name\": \"email\",\n",
    "              \"comparison_vector_value\": 1,\n",
    "              \"label_for_charts\": \"Levenshtein <= 2\",\n",
    "              \"log2_bayes_factor\": 7.800719512398763,\n",
    "              \"m_probability\": 0.3019669634613132,\n",
    "              \"sql_condition\": \"levenshtein(\\\"email_l\\\", \\\"email_r\\\") <= 2\",\n",
    "              \"u_probability\": 0.0013542812658830492\n",
    "            },\n",
    "            {\n",
    "              \"bayes_factor\": 0.10692840956298139,\n",
    "              \"cl_id\": \"email_0\",\n",
    "              \"comparison_name\": \"email\",\n",
    "              \"comparison_vector_value\": 0,\n",
    "              \"label_for_charts\": \"All other comparisons\",\n",
    "              \"log2_bayes_factor\": -3.225282884575804,\n",
    "              \"m_probability\": 0.10654901125069259,\n",
    "              \"sql_condition\": \"ELSE\",\n",
    "              \"u_probability\": 0.9964518474197885\n",
    "            }\n",
    "          ]\n",
    "        },\n",
    "        \"layer\": [\n",
    "          {\n",
    "            \"encoding\": {\n",
    "              \"color\": {\n",
    "                \"field\": \"comparison_name\",\n",
    "                \"legend\": null,\n",
    "                \"type\": \"nominal\"\n",
    "              },\n",
    "              \"tooltip\": [\n",
    "                {\n",
    "                  \"field\": \"comparison_name\",\n",
    "                  \"type\": \"nominal\"\n",
    "                },\n",
    "                {\n",
    "                  \"field\": \"label_for_charts\",\n",
    "                  \"type\": \"nominal\"\n",
    "                },\n",
    "                {\n",
    "                  \"field\": \"sql_condition\",\n",
    "                  \"type\": \"nominal\"\n",
    "                },\n",
    "                {\n",
    "                  \"field\": \"m_probability\",\n",
    "                  \"type\": \"quantitative\"\n",
    "                },\n",
    "                {\n",
    "                  \"field\": \"u_probability\",\n",
    "                  \"type\": \"quantitative\"\n",
    "                },\n",
    "                {\n",
    "                  \"field\": \"bayes_factor\",\n",
    "                  \"type\": \"quantitative\"\n",
    "                },\n",
    "                {\n",
    "                  \"field\": \"log2_bayes_factor\",\n",
    "                  \"type\": \"quantitative\"\n",
    "                }\n",
    "              ],\n",
    "              \"x\": {\n",
    "                \"field\": \"log2_bayes_factor\",\n",
    "                \"scale\": {\n",
    "                  \"domain\": [\n",
    "                    -10,\n",
    "                    10\n",
    "                  ]\n",
    "                },\n",
    "                \"title\": \"Comparison level match weight = log2(m/u)\",\n",
    "                \"type\": \"quantitative\"\n",
    "              },\n",
    "              \"y\": {\n",
    "                \"field\": \"cl_id\",\n",
    "                \"sort\": \"-x\",\n",
    "                \"title\": \"Comparison level\",\n",
    "                \"type\": \"nominal\"\n",
    "              }\n",
    "            },\n",
    "            \"mark\": {\n",
    "              \"type\": \"bar\"\n",
    "            }\n",
    "          },\n",
    "          {\n",
    "            \"encoding\": {\n",
    "              \"text\": {\n",
    "                \"field\": \"comparison_name\",\n",
    "                \"type\": \"nominal\"\n",
    "              },\n",
    "              \"tooltip\": [\n",
    "                {\n",
    "                  \"field\": \"comparison_name\",\n",
    "                  \"type\": \"nominal\"\n",
    "                },\n",
    "                {\n",
    "                  \"field\": \"label_for_charts\",\n",
    "                  \"type\": \"nominal\"\n",
    "                },\n",
    "                {\n",
    "                  \"field\": \"sql_condition\",\n",
    "                  \"type\": \"nominal\"\n",
    "                },\n",
    "                {\n",
    "                  \"field\": \"m_probability\",\n",
    "                  \"type\": \"quantitative\"\n",
    "                },\n",
    "                {\n",
    "                  \"field\": \"u_probability\",\n",
    "                  \"type\": \"quantitative\"\n",
    "                },\n",
    "                {\n",
    "                  \"field\": \"bayes_factor\",\n",
    "                  \"type\": \"quantitative\"\n",
    "                },\n",
    "                {\n",
    "                  \"field\": \"log2_bayes_factor\",\n",
    "                  \"type\": \"quantitative\"\n",
    "                }\n",
    "              ],\n",
    "              \"x\": {\n",
    "                \"field\": \"log2_bayes_factor\",\n",
    "                \"scale\": {\n",
    "                  \"domain\": [\n",
    "                    -10,\n",
    "                    10\n",
    "                  ]\n",
    "                },\n",
    "                \"title\": \"Comparison level match weight = log2(m/u)\",\n",
    "                \"type\": \"quantitative\"\n",
    "              },\n",
    "              \"y\": {\n",
    "                \"field\": \"cl_id\",\n",
    "                \"sort\": \"-x\",\n",
    "                \"title\": \"Comparison level\",\n",
    "                \"type\": \"nominal\"\n",
    "              }\n",
    "            },\n",
    "            \"mark\": {\n",
    "              \"align\": \"right\",\n",
    "              \"dx\": 0,\n",
    "              \"type\": \"text\"\n",
    "            }\n",
    "          }\n",
    "        ]\n",
    "        }\n",
    "      ```\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Edit in Vega-Lite\n",
    "\n",
    "Opening the JSON from the above chart in [Vega-Lite editor](https://vega.github.io/editor/#/url/vega-lite/N4IgJAzgxgFgpgWwIYgFwhgF0wBwqgegIDc4BzJAOjIEtMYBXAI0poHsDp5kTykBaADZ04JAKyUAHJQAMlAFYQ2AOxAAaEFBUAzGmTShiNOAHcDmlZhrKGbBhAAScPVjQBmGTI1blVm3YgAdRoAE3p3TwBfSI0QpEwUVFBlJAQ4NBA4hP4kADYoSQAWJABOKG0ARiQZKAB2JAAmKBCPKCgqmTgmiorJMRAYzPikCDhMCHMsgTyC4rLK6rrG5tb26q723v7UAG1QJiQATzgIAH1tJChMNgAnNElcylyGksKSktySsVqxEslJbyCU6hDK6G4QTCnFJpU6FdQWBA4JA3GhKZRQ1LpdBgiEYtLwrSI5GolSnUhXW5kpCCBhYwoaQRIJhwIHaSmwZHjDIAUQAHpdMAACZCYWCCnGQ6HpBlsMgNU4HY5nC4Uu6oR6FNy5WoPQqSGRuBoyf4VDQIU44G5sA5MGjCTCHNByMQyXqvColCq5Cq1NySCr0kAQACOQJ8ITo7FU6AAOiAJXi4KdBHHBQBeQVxhNS043OPwhgWq02u10R2oOSeb4lNzfCpuEo-A1uWoyMQxfZHE7nAW3e4NKQ+zViMQ+yQtwoNQHAkKgmjgyWY05uAlsIkotGJucLrfeNdIjek8nXG5UmlYtwMpks87smCciboAAimJuSELgjgpGUECwzmUgoADwZhU8KCLK8qKt2Konmgjwtr0zzPIUYhuGhLxmkW1pMqWDpOpQHq-NWPqNqhFSumIGghmGKgRlYKgZHEaRvh+X5wD+f7WAAFFm864jmKbqJm8Z8YuMJ5iAACUQEgQWWElva5aVjIDRat6moNL6KF6m4HYgFByq9mqtaUIUOoVA0hS5A8OqaQCmhAiC2KiYmpwNKu64kuiUrbvxmIeQeXlknAqpnrSaBTiAjLMqyd4PhkABSyJsKcJjWAA1p+p4QDQCB2sSDqCgAfBmcglGBEEKl2hmqmgEgBv83xFP6hSurkl4gOalrYbain4TIKF-KhBpIc2xqUUGoanOGkYMeg8jJalGVZacOV5YyKIOjxIk7gJcZqMJ2ZLhJ0klYKZVyd1Cllv1rovN6vxqa2ki1BZekGT2tWoD0lANLk-2FPWrYqapYiRVAjmzs5u1LqBe6eZuPnQ35+Lw4Fm7HpSxDUuF31XjFt6nhyNxcugSVWktyiZXA2W5flm2HMVpVSBVcpVUqn2wagbimTWQ25K6mrjj8mFXThfUVrI7UfL6KnfG8bgVGIgbUdNtGzdGIALRTaVUyta302W21HTCgkHbxMPiXGp3M5Il3FuLN2S1W2o-OOfwNEDNaSO91Wc32kuEQ1I7-TLDZiLk05OTtKNJjIAXEoj-nI2J0oIujR4hSeYVYl4UXXrFRP3iTj4gAAgoIgiCmw9A04KhIZz+rOQX7MEB-wA61r6I4eJ7LyqYGXUO71TtyB6ntfA0RRGu1mqvFRU0zfRmvcgAMgAyty9s9bhSmUH8er-crHr6oRDS+xzbdqj0PMqf6-oelWmqmg5M4ZBADA3DmcJo4npJI0GT+Plf6HnRJjU82NzxoEDNFG8bIi7xXQHyAUwp4hig-l-fyMo2YfSvnBKQRpUIvQsvqbUJRIpDx3hLZ0YNWwuknG8QGZEF40WUHRKMGQ4wYL2iAdMwluHHXzBoQsYsR54WdgNf47wfgjjug0SyJQL7QSMvcOQ3payET6MaT2gYIZv3QAImEK4QFBQAYYtODc-5gKzljHGF58ZwLiiXDIL5mLvmTGxDimB-wyUFHDKKlVcEqPVJQdCrx3gyHajqP4L9KHXXEXIeRfcfiqSNF8dqDwWFqzYRrRir53Gfm-L+bx3EuFAKXGbfh5SrZSV8f4kRw9d63U0iOKyll6xekBpZXISiapcz1JQb4EcigCz1GIf0E09HR3MW5BOoCtwGOqRY-cVjgqhUgbjSKsDC7TWLqTEA5MUq62prTdaBVGZnQutglul9gkSAFp7ZCoN-pGg6nEx2CTZBvEFjWP0HozI9CyUvDh81FrHP1nTDaRsymYNNvtKpsKkwnSZudfe294l708ORGQJQBoNG+H6NwZk3g+zUJ2W5X16ymR1JIGekhGy1BQl6KOUNAGItOP4yx8yzFLLmUFcBOc0Av22YTXZiCDlguWjTVakLzkorkHba57NlFfR5ik54qlWx+jvoPeSHzMUujatEh4tZPgoSBerZeGRtZHKlacw2W0YU8PNmynMyLLks2EXqsRmKDSvRkLUay7x-n4oeL0-2aox6aVegrQNJr5GR1ftMpZpx44mKTviRZiK+UYxsRAuxToHE7OJvsiuVca7wBuPXFZoCJhKqCV9Du+9PieG0QGg0AaSii0adQgif16x1kifWSylkLU5KtUgjeW8vWiKaZLP4gaiVNjMjirUukyX6VbsE1SjwdQaMPvqWojLwaQ0YtaU4-R03-2Tpka0ObM7rILagCaIr4FiucUg-kVxUGihgM3ZVfSA61CkC9Bh455EqRrLE71c65CA0JTqZWLUzKBsTarYFc0QBxhCOewSfDsPnokui-Vt0hxHpQxR8jtRFEbobVzXIEhGXPDBtZIlqFfgsrPUwWE97vI3pw0wXjazs4bLpEW0VJbS6uJpgUzxxSfHAT8f+ujAdHgGn+FqQNU9JwvO7VQ0eBFjSRKJeRRWDGzJErHewzDTEZOsSKZxZQ20BPJnhQR7jyLFP1Jg72rFuRNT+a1AF4LWpw14NQC8al1kBa-EibilCHUpmspc8Y9OqyAECaEwK0T7hxNvskxkYI9BrB+OFJYP99at1fUKCBpW5FGyFEnPqGsemMX4Rvp4b2ysUPkNHZNVh1nNYxgAoKUbY3xuCiZBALiWQkwRm0NoLiAByBA5WltqAhDcHAVg0jOdw-tYbE2jtjaWwAUgAJr8FOwgK7IQluSQ25gLbO24B7Y8wdkbx2Jtncu9d2793JLSS84d47xGfW3UJZD8cUOiVheCTzBsMiAwPN+AGXRp70Aufclevjmbb2CZx8J2xUCIt5acfsorMASsVEFMcZEymqtcwqJQQNjWvitX7e7chrWSOB1qEaSJEdeuGnpUeqzuTYyfa+5Npg03ZunHm4tpbdObjrc29t3Kr33OufUCD6XJ2LtXZu6du7D31cvbe7mD7+vxs-aN-9wHQOQJ64m2D2DsgXRIS939L367yUqq5mov6R73T85eL6f4nHMfns5TW0x-G72E+y0+4VBcJN7NLpT6nMhadwGRHWgJODGftzUb6RskHjSvW9LSnn4PA64trGDDwgNqN0vFxOrDUuvtTZm-EObNAFvLZV2rp7Gvdva7Ni7-Xdu-sm-u4957mvLcSTUFP6XM-jem8B3UmQa-Rtu97R4FsfqT-H5bHDr6ylJz896B26NAs2xR-x6moTGXE9pfmcnknedX3k9LmW6uWuKtLlLyAvcCIvClLmfgYDF6Y0HoRlelcZU+Wvd3TwTSFpN4EPRsVJCadDS1EFEANeTeA-AzDJRrcgig8g9sWjYva+MqAab2RqV4aHRLDHTQMsDlV-G9KAG6JPPNQVPGfOAmfLDPHkL9IUEUWABnSAgOHmRrFtBlD4cZBjD4FA3tAWesfUelIlTScvF+PA8dAguMHgh0HXfDdg0wojGdHtAzNsUcMyOsW-AMVCC-QPUyT2RqAaM+D4d4dHfRCww4F-QnABEwx0Pgx9b-MnBBD9cuSuQAytatBGFQMAwJWgtAfgZnSyF0cyIZb0bAiad5OvOQBsB4D0ciBjfnSDLtfrbJQbHkKdEgz5V4MZEdE0IaMyVwgOP6EoJ4ANIWBhQWF+JLDIRAJAO0WZYIm9UYu0LLfgnLUnIQxxaI-ZZBb9SQirQvG5APAOaQPoqvb5EcHUXVWdXtVHPUDnMGF6XwjqAwuo2MEAaYoEPDDMOMR4q3EARo31CeP0V6RWDSccB5TotUJJfeG-N0JWRvVg-wt42PJI3HLEB45AGY8IkTFPKI99fZVeOTRzXxbHTYgDCNNAGAzwV6L4CyEoxdNQgzDwD0UOWedRYddvAgwpdieTUpREsYp4tzDk8YzzDMPEhpfTT5LFWsJrCyBjDTBg8+GgmQyNQzT4bTHFCOF4f0BsJ-N4tND-ePPHR42YiI3GH-NPEQ8VAAitOuEAtEFIiA7YtUfgHmeRS4qeIoIZfUY4mwz5co5WXFHofFOLfFao24iXQgho6woUved4fzUcIoRlD0GldsAAXUGEZGODVD2AeOUC0AjGUH0CSAsHAjVFAF0BZFZQtOvVRiinIHYihhsErg0AdBwAROUDXGsGpAGDrLYDYEECsBwDQDTKLMEBLLjwzTTnrMbObJSEEDbMLOMAHIyF-2WIL1HIyCbLygnKnJEmLPfkXnwIYjrMOAbOXPHNbL0n7NZUKKaT3IPPQGDAYCQF8DoHiBoFIHXNPIyEFLa0vIRJvLvKsASCsGfJPJnNZRUzuE-IyG-PvL-KfPSEAs3PQHAK2MA1ApACXOvNvMgsfIAvjI0F5HMFfPgtSNlPhGgGpCxFABwyROjB2AyK8HIkTLrLoE-AyAAGEhyVBBQWSq51jBQTBnAyAsA+EEKuIEACAGBJJ4RUKQAILfzMKYKNByxpy4Kk1ZwqJbhMAMh+BcKGLMAmL0BWK4SOK2JJywL0AVyWzJzogzRkR0pzBJKDg7hLLQB2JMzrAczQBvFeR1Lcz8KtThyJL9yxzVzjz2zOzuzezFLZz0BSz4T-KryQAzK1zYLIrFji1RCTL4qjyLKN0fLAyrV0qErgqIqzyfNeCUKArwL0KZL-y5Kiq3ySq8J0rpKHzqqXygKMgQLYqvzKrmroLWqlKEKCS8FGruqoKsKcK8K2qCLrSkLiKoBSLJg1wxiqKaK1A6LBhfzdKQB9LG5DLSAuK0EYAeK+KBKMwhKRKxLOqKqfyeqAL5KJqlLhjVKSYNKtKULGKETtqrFdqWRLrTLMqBhBhkAbgbLczqQ9BNYUR+L1LYhcKKx0qPL1LohEygA), it is now behaving as intended, with both bar and text layers sorted by match weight.\n",
    "\n",
    "If the chart is working as intended, there is only one step required before saving the JSON file - removing data from the template schema.\n",
    "\n",
    "The data appears as follows with a dictionary of all included `datasets` by name, and then each chart referencing the `data` it uses by name:\n",
    "\n",
    "```\n",
    "\"data\": {\"name\": \"data-a6c84a9cf1a0c7a2cd30cc1a0e2c1185\"},\n",
    "\"datasets\": {\n",
    "  \"data-a6c84a9cf1a0c7a2cd30cc1a0e2c1185\": [\n",
    "\n",
    "    ...\n",
    "\n",
    "  ]\n",
    "},\n",
    "```\n",
    "\n",
    "Where only one dataset is required, this is equivalent to:\n",
    "```\n",
    "\"data\": {\"values\": [...]}\n",
    "```\n",
    "\n",
    "After removing the data references, the template can be saved in Splink as `splink/files/chart_defs/my_new_chart.json`"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Combine the chart dataset and template\n",
    "\n",
    "Putting all of the above together, Splink needs definitions for the methods that generate the chart and the data behind it (these can be separate or performed by the same function if relatively simple).\n",
    "\n",
    "### Chart definition\n",
    "\n",
    "In [`splink/charts.py`](https://github.com/moj-analytical-services/splink/blob/master/splink/charts.py) we can add a new function to populate the chart definition with the provided data:\n",
    "\n",
    "```python\n",
    "def my_new_chart(records, as_dict=False):\n",
    "    chart_path = \"my_new_chart.json\"\n",
    "    chart = load_chart_definition(chart_path)\n",
    "\n",
    "    chart[\"data\"][\"values\"] = records\n",
    "    return altair_or_json(chart, as_dict=as_dict)\n",
    "```\n",
    "\n",
    ">**Note** - only the data is being added to a fixed chart definition here. Other elements of the chart spec can be changed by editing the `chart` dictionary in the same way. \n",
    ">\n",
    "> For example, if you wanted to add a `color_scheme` argument to replace the default scheme (\"tableau10\"), this function could include the line: `chart[\"layer\"][0][\"encoding\"][\"color\"][\"scale\"][\"scheme\"] = color_scheme`\n",
    "\n",
    "### Chart method\n",
    "\n",
    "Then we can add a method to the linker in [`splink/linker.py`](https://github.com/moj-analytical-services/splink/blob/master/splink/linker.py) so the chart can be generated by `linker.my_new_chart()`:\n",
    "\n",
    "```python\n",
    "from .charts import my_new_chart\n",
    "\n",
    "...\n",
    "\n",
    "class Linker:\n",
    "\n",
    "    ...\n",
    "\n",
    "    def my_new_chart(self):\n",
    "        \n",
    "        # Take linker object and extract complete settings dict\n",
    "        records = self._settings_obj._parameters_as_detailed_records\n",
    "\n",
    "        cols_to_keep = [\n",
    "            \"comparison_name\",\n",
    "            \"sql_condition\",\n",
    "            \"label_for_charts\",\n",
    "            \"m_probability\",\n",
    "            \"u_probability\",\n",
    "            \"bayes_factor\",\n",
    "            \"log2_bayes_factor\",\n",
    "            \"comparison_vector_value\"\n",
    "        ]\n",
    "\n",
    "        # Keep useful information for a match weights chart\n",
    "        records = [{k: r[k] for k in cols_to_keep}\n",
    "                   for r in records \n",
    "                   if r[\"comparison_vector_value\"] != -1 and r[\"comparison_sort_order\"] != -1]\n",
    "\n",
    "        return my_new_chart(records)\n",
    "\n",
    "```\n",
    "\n",
    "\n",
    "## Previous new chart PRs\n",
    "\n",
    "Real-life Splink chart additions, for reference:\n",
    "\n",
    "- [Term frequency adjustment chart](https://github.com/moj-analytical-services/splink/pull/1226)\n",
    "- [Completeness (multi-dataset) chart](https://github.com/moj-analytical-services/splink/pull/669)\n",
    "- [Cumulative blocking rule chart](https://github.com/moj-analytical-services/splink/pull/660)\n",
    "- [Unlinkables chart](https://github.com/moj-analytical-services/splink/pull/277)\n",
    "- [Missingness chart](https://github.com/moj-analytical-services/splink/pull/277)\n",
    "- [Waterfall chart](https://github.com/moj-analytical-services/splink/pull/181)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
